I actually looked for kosher guava paste for a long time. I ended up finding only one brand in Mexico that had the VK-aleph-aleph Mexican orthodox hecksher, and they only had it in the kosher store I went to. I believe the brand is Guayaco but I will have to double-check. Maybe it will just be easier to make your own. For a quick shortcut, if you don't have access to fresh guava, you can buy the guava nectar from JUMEX (in the Mexican section of your supermarket) which is also heckshered and maybe add some fruit pectin to solidify it. Good luck.
It was actually Guayeco. You can see it at:
It's called "Ate de Guayaba" (pronounced ah-tay). It is a Mexican site but maybe you can ask them who distributes it in the US (the label is in both Spanish and English, so I assume they export it to the US).