Guidance for combining webhooks and topics

I’m making a webhook handler for GitHub webhooks that pushes incoming webhooks to a topic.

import (
	"encore.dev/pubsub"
	"github.com/go-playground/webhooks/v6/github"	
)

var Webhooks = pubsub.NewTopic[*github.PushPayload]("webhooks", pubsub.TopicConfig{
	DeliveryGuarantee: pubsub.AtLeastOnce,
})

This code doesn’t work:

.PushPayload is not a supported type in Encore
	note: you can only use types defined within your Encore app and builtins
	builtins also include time.Time, json.RawMessage, and encore.dev/types/uuid.UUID

I have a few questions about this.

  • Can this limitation be lifted at some point?
  • If so, do you think that might happen soon?
  • What do you think I should do in this situation, assuming I just want to push the incoming webhook JSON body to the topic (and react to it elsewhere, preferably using this same type)?
  • Would vendoring the dependency work (is the vendor directory “within your Encore app”)?

Hey @wolverian and welcome!

We definitely want to lift this restriction. The main challenge is performance. It’s slightly tricky to maintain a very fast static analysis pass while parsing all external dependencies. We have a plan for how to solve this but it will take a little bit of time, perhaps a few months, before we have time to work on that.

In the meantime I think the simplest solution is to define your own event type and pass the payload using json.RawMessage. Something like this:

type GitHubWebhook struct {
    PushPayload json.RawMessage
}

func marshalPushPayload(event *github.PushPayload) (json.RawMessage, error) {
    bytes, err := json.Marshal(event)
    return json.RawMessage(bytes), err
}

I think vendoring the dependency should work as well if you prefer that solution.

1 Like

Hey Andre, and thanks for the welcome!

If I understand your suggested solution right, my topic would have the type Topic[*GitHubWebhook], and I would need to unmarshal the wrapped payload in all topic subscribers?

I think I prefer vendoring to that actually. :smile:

Thank you for the suggestions - I’ll play around and see what works best here. I understand there are technical limitations here and roadmaps are what they are.

1 Like