What material is the 1" spacer? I had some problems with a Phenolic spacer and the Offy carb plate. That was solved by going to an Aluminum spacer.
If you have been running a phenolic spacer, check the Offy carb plate to see if it's true. Mine warped under the phenolic.
Another was an internal leak through the four screws that hold the plate to the manifold. RTV across the screw heads once they were tightened solved that one.
Vacuum lines and PCV hose is a possibility. A loose or cracked hose might exhibit that sort of issue.
With the Carter AFB and no choke I have to baby my engine 'till it gets some heat in it. Until it's warm, it won't idle. Idle will change slightly when engine temperature varies, though not to the extreme your experiencing. Perhaps 50 rpm normal to hot.
If you have the idle up high enough to be right on the edge of unmasking ported vacuum, you may be adding vacuum advance inadvertently. The timing would tend to bump up. See what it does with your light. Disconnect the Vacuum advance and see if it still does the same thing.
How about a loose advance spring perch? Once again, you'd likely see erratic timing. With a mild build, you should be able to idle down a bit. Mine idles nicely a tad over 600 rpm in gear now.

Hope you find it Seymore!

CJ

_________________
Part of Tyrde-Browne Racing