Thanks for the feedback @Noah-Sager . I unfortunately do not have a public service that I can share with this issue.
I'll have to dig back into this and validate the information provided, but I believe the problem we were seeing is that we were having to use the SubLayer in the export call to even make the annotation be drawn rather than the parent layer. Although it's possible we just use the sublayer instead of the parent layer because that's how we thought it was supposed to be done?
It the example you provided I noticed you had two sublayers defined. I guess by using the parent layer you get all of the sublayers drawn. In our case we only have a single sublayer defined "Default", so that isn't an issue for us, not sure if it would ever be an issue for others.
I thought I had tried changing the layer list to use the parent instead of the sub layer before posting this question, but a quick test via fiddler on our private service seems to suggest this should work for us.
Thanks for the info, hopefully this will enable us to solve the problem.