Adding prism path to a main objective can affect it in various ways. It can add SA and CA, and it can also be corrective somewhat for CA of the objective. There is no set rule of thumb though to determine what the effect will be. As Tom Back noted on this topic, the only way you will be able to tell is if you model the specific objective with the added prism path to see if anything positive or negative may happen, and if it does then if it will sneak into the visual threshold or not.
As far as the contrast differences you noted that is a no brainer as all the dielectrics I have run through my scopes always showed marked more scatter than prisms. The only scatter you are going to get off the prism is from the two air-glass surfaces and whatever negligible amount from the light path through the prism. 40 layers of dielectric coatings however are a whole other story and each layer adds its scatter and any imperfection or micro-dust between the layers will wreak havoc. Only the most very expensive premium ones come close to how little scatter a quality prism shows, like the AP and Baader Maxbrights. But even these still show more than a quality prism.
As far as those orange highlights, I suppose, depending on how the orange spectrum was corrected in your main objective that a prism path might shift it in the opposite direction some, but only way to tell if that might be happening is to know the prescription of the main objective and model that with a prism's prescription. While I have, with my scopes, experienced very minimal addition of CA when prism path is added, and SA, it never reached the visual threshold though unless the target was glaringly bright and the magnification was up near 40-50x/in at least. When I add a binoviewer to a large prism diagonal (2") then SA reaches the visual threshold at planetary magnifications so when I do that I will either use the short path Baader Zeiss 1.25" prism or use a mirror diagonal so the SA does not reach an impactful visual threshold with my scopes.
What I would be interested in hearing though is if that orange CA you are seeing is there if you view straight through without any diagonal also. At least doing this would let you know if it is something about the particular diagonal or not. Btw, I assume that when you see this color it happens when the crater in directly on-axis in the eyepiece so that lateral color from the eyepiece is not the issue.