How to integrate REST APIs with EventBridge? A demo setup to get you started.

AWS EventBridge is a serverless event bus service that makes it easy to ingest, filter, and route events from various AWS services and custom sources. It allows you to build event-driven applications and architectures. But how to integrate REST APIs with Eventbridge? Let’s figure that out in a demo!

The theory

There are several widely recognised methods for sending events to an EventBridge bus:

• AWS Service Integrations
  Events can automatically be sent to the EventBridge bus when specific occurrences take place within your AWS environment, e.g. when a server scales up.
• AWS Partner Integrations
  EventBridge offers integrations with external partner SaaS products like Salesforce, DataDog, Zendesk, and more. This allows events from these services to be seamlessly forwarded to your EventBridge bus.
• Custom Events
  You can create and publish custom events to the event bus using the EventBridge API. This involves writing custom code, using the AWS SDK to create and publish events to your specific requirements.
• Cron-Based Events
  Events can also be scheduled and triggered at specific times using cron expressions, enabling you to automate actions at precise intervals.

A much less known, but often very valuable integration, is the direct integration between API Gateway and EventBridge. This integration provides the capability to treat a REST API event as an event message by publishing it to an event bus and subsequently using it within an event-driven workflow.

The theory in practice

In this demo setup, we will create a REST API with a POST method that enables users to submit car details to the API. These details will be validated, transformed, and then published to our Event bus. Depending on the content of the body, it will be routed to the correct SQS queue for further processing.

You can find all code for this demo in the dropdowns.

curl --location --request POST https://<ID>.execute-api.eu-west-1.amazonaws.com/prod/CarDetails --header 'Content-Type: application/json' \
--data-raw '
{
    "items":[
        {
            "CarData":"{\"data\":\"Audi A4 is created\"}",
            "BrandType":"audi",
            "Source": "car_data"
        }
    ]
}'

To deploy this solution, we use AWS SAM. This is an extension of CloudFormation, which is an AWS native Infrastructure as Code (IaC) service. The extension offers better integration with serverless services such as API Gateway and Lambda. The primary focus of this blog is not on SAM, but it serves as a more convenient setup method for me compared to “ClickOps.”

The full template is available here and can be deployed using certain commands.

$ sam build
$ sam deploy --stack-name api-gw-eventbridge-demo --capabilities CAPABILITY_IAM

Let’s take a closer look at our template. The most important part is our API method configuration.

  apiGatewayRootMethod:
    Type: AWS::ApiGateway::Method
    Properties:
      AuthorizationType: NONE
      HttpMethod: POST
      MethodResponses:
        - StatusCode: 200
          ResponseModels:
            application/json: Empty
      Integration:
        IntegrationHttpMethod: POST
        Type: AWS
        Credentials: !GetAtt ApiGatewayRole.Arn 
        Uri: !Sub arn:aws:apigateway:${AWS::Region}:events:action/PutEvents
        PassthroughBehavior: WHEN_NO_TEMPLATES
        RequestTemplates: 
          application/json: !Sub 
            - |- 
              #set($context.requestOverride.header.X-Amz-Target = "AWSEvents.PutEvents")
              #set($context.requestOverride.header.Content-Type = "application/x-amz-json-1.1")            
              #set($inputRoot = $input.path('$')) 
              { 
                "Entries": [
                  #foreach($elem in $inputRoot.items)
                  {
                    "Detail": "$util.escapeJavaScript($elem.CarData).replaceAll("\\'","'")",
                    "DetailType": "$elem.BrandType",
                    "Source": "$elem.Source",
                    "EventBusName": "${EventBusName}"
                  }#if($foreach.hasNext),#end
                  #end
                ]
              }
            - { EventBusName: !Ref EventBus }
        IntegrationResponses:
          ...
      ResourceId: !Ref apiGatewayResource
      RestApiId: !Ref apiGateway

For the integration URI, we configure arn:aws:apigateway:${AWS::Region}:events:action/PutEvents, linking our API Gateway POST method with EventBridge. Next, we will use the Velocity Template Language (VTL) to craft a mapping template. This template is designed to transform our request body and align it with the required request structure for the EventBridge PutEvents API.

{
   "Entries": [ 
      { 
         "Detail": "string",
         "DetailType": "string",
         "EventBusName": "string",
         "Resources": [ "string" ],
         "Source": "string",
         "Time": number,
         "TraceHeader": "string"
      }
   ]
}

Our input items (as seen in the example request) will be mapped to conform the structure above. We will transform our list of items to a list of entries.

{ 
   "Entries": [
      #foreach($elem in $inputRoot.items)
        {
          "Detail": "$util.escapeJavaScript($elem.CarData).replaceAll("\\'","'")",
          "DetailType": "$elem.BrandType",
          "Source": "$elem.Source",
          "EventBusName": "${EventBusName}"
        }#if($foreach.hasNext),#end
        #end
  ]
}

Thanks to the EventBusName property, AWS will automatically determine to which bus this event should be forwarded. The EventBusName will be automatically propagated by our SAM template. Our Event Bus is created within that same template and is referenced by our mapping template.

The API and its mapping template in AWS

As we now understand the API Gateway – EventBridge integration, we can check our EventBridge rules. EventBridge rules are rules created on a bus, defining how specific events should be routed to particular targets based on the content of the events.

  AudiEventsRule:
    Type: "AWS::Events::Rule"
    Properties:
      Description: "Rule to capture Audi events and send to AudiQueue"
      EventBusName: !GetAtt EventBus.Name
      EventPattern:
        source:
          - "car_data"
        detail-type:
          - "audi"
      Targets:
        - Arn: !GetAtt AudiQueue.Arn
          Id: "AudiQueueTarget"  

If the event has “car_data” as its source and detail-type matches with “audi”, it will be directed to the Audi SQS queue. A similar rule is established for the detail-type of “bmw.” If it doesn’t match either of these criteria, the event is discarded and not routed anywhere.

To test the complete setup, we have to execute a POST request to our API.

curl --location --request POST https://<ID>.execute-api.eu-west-1.amazonaws.com/prod/CarDetails --header 'Content-Type: application/json' \
--data-raw '
{
    "items":[
        {
            "CarData":"{\"data\":\"Audi A4 is created\"}",
            "BrandType":"audi",
            "Source": "car_data"
        }
    ]
}'

This request includes all the attributes necessary for our mapping template to execute the correct transformation. In the logs we can see both our initial and our transformed request.

The payload is forwarded as an event to our Event Bus. We have two rules created on our bus, and our event matches one of these rules:

• It has detail-type audi
• It has source car_data

Our rule will forward the event to our Audi SQS queue.

Our Event Bus and its rules

Of course, you can create much more advanced rules, and you can also enhance your event by adding extra data using EventBridge pipes, and even more. In fact, this is just the beginning of the truly exciting event-driven capabilities.

Now let’s retrieve messages from our Audi SQS queue. That queue is configured as target of our Audi EventBridge rule. We see there is one message available on the audi-queue.

Poll the message and check the content.

There it is! The payload from our POST request is now a message ready for use in an event-driven workflow. AWS provides numerous integrations with EventBridge, enabling you to initiate Step Function workflows or use Lambda Functions for further event processing.

Conclusion

In theory

This setup teaches us that integrating REST APIs with EventBridge is straightforward. We don’t need to write or maintain custom code, servers or Lambdas. We can convert the payload of a request into a message that is suitable for any event-driven workflow.

In practice

This implies that clients don’t need to write custom code or use the AWS-specific SDK to publish events to EventBridge. They can simply integrate with a commonly used HTTP endpoint. They don’t need to be familiar with the AWS-specific details but can still take advantage of all the benefits that event-driven flows offer in AWS.