AWS API Gateway and Lambda

API Gateway and Lambda

The most common serverless pattern on AWS, and the source of one error more than any other: {"message": "Internal server error"} with a 502, caused not by your code failing but by returning the wrong shape.

This page is mostly about getting that shape right.

Proxy integration

With AWS_PROXY, API Gateway passes the entire request to your function and expects the entire HTTP response back. Your function handles routing, status codes and headers.

aws apigatewayv2 create-integration --api-id "$API_ID" \
  --integration-type AWS_PROXY \
  --payload-format-version 2.0 \
  --integration-uri "arn:aws:lambda:eu-west-1:123456789012:function:orders" \
  --timeout-in-millis 29000

The alternative, non-proxy AWS integration, uses mapping templates to shape both directions. It lets you keep transformation out of your code, at the cost of VTL in your infrastructure config and error mapping that depends on matching error strings. Use proxy unless you have a specific reason not to.

The response shape

This is the whole ball game. A proxy integration requires an object with specific keys, and anything else is a 502:

def handler(event, context):
    return {
        "statusCode": 200,
        "headers": {"Content-Type": "application/json"},
        "body": json.dumps({"orderId": "A-4821"})
    }

Three rules that cause nearly every 502:

body must be a string. Returning a dict is the single most common mistake, json.dumps() it.

return {"statusCode": 200, "body": {"ok": True}}        # 502
return {"statusCode": 200, "body": json.dumps({"ok": True})}   # correct

statusCode must be present and an integer. "200" as a string fails.

An unhandled exception is a 502, not a 500 with your error. Catch and return deliberately:

def handler(event, context):
    try:
        return {"statusCode": 200, "body": json.dumps(do_work(event))}
    except NotFound:
        return {"statusCode": 404, "body": json.dumps({"error": "not found"})}
    except Exception:
        logger.exception("unhandled")
        return {"statusCode": 500, "body": json.dumps({"error": "internal"})}

The gateway's error message never tells you which of these it was. The function's CloudWatch log does, which is why the log group is the first place to look, not the gateway.

Payload format 1.0 vs 2.0

HTTP APIs default to 2.0, REST APIs are always 1.0, and the event structures differ in ways that break code moved between them:

That last row is a real trap. Under 2.0, a function returning a plain object with no statusCode is treated as a successful JSON body with status 200, so a function that silently returns the wrong thing looks like it works. Under 1.0 the same return is a 502. Set statusCode explicitly either way.

Header names arrive lowercased, and event["headers"] in 2.0 joins multiple values with commas rather than providing a multi-value map.

Pin the format explicitly rather than relying on defaults:

--payload-format-version 2.0

Permissions

API Gateway needs permission to invoke the function. This is a resource-based policy on the Lambda, not an execution role:

aws lambda add-permission --function-name orders \
  --statement-id apigw-invoke \
  --action lambda:InvokeFunction \
  --principal apigateway.amazonaws.com \
  --source-arn "arn:aws:execute-api:eu-west-1:123456789012:$API_ID/*/*"

Creating the integration in the console adds this automatically; creating it with the CLI, Terraform or CloudFormation does not, which is why a hand-built integration fails with:

Execution failed due to configuration error: Invalid permissions on Lambda function

Keep --source-arn scoped to your API. A wildcard principal with no source restriction lets any API Gateway in any account invoke your function.

CORS

Handled in two places, and getting it half right is worse than not doing it at all.

Preflight (OPTIONS), on an HTTP API this is configuration, and the gateway answers without invoking Lambda:

aws apigatewayv2 update-api --api-id "$API_ID" \
  --cors-configuration 'AllowOrigins=https://app.example.com,AllowMethods=GET,POST,PUT,DELETE,OPTIONS,AllowHeaders=content-type,authorization,AllowCredentials=true,MaxAge=300'

Actual responses: your function must still return the headers, because CORS applies to every response, not just the preflight:

CORS = {
    "Access-Control-Allow-Origin": "https://app.example.com",
    "Access-Control-Allow-Credentials": "true",
}
return {"statusCode": 200, "headers": {**CORS, "Content-Type": "application/json"}, "body": json.dumps(data)}

The classic symptom is a request that works in curl and fails in the browser with a CORS error, because the error response lacks the headers even though the success response has them. Return CORS headers on error paths too, or a 500 becomes an unhelpful CORS message that hides the real problem.

AllowCredentials=true cannot be combined with AllowOrigins=*; browsers reject it.

Timeouts

Two limits, and the smaller one wins:

• API Gateway: 29 seconds, a hard maximum on REST APIs. HTTP APIs also default to 30 seconds.
• Lambda: up to 15 minutes.

So a Lambda configured for five minutes behind API Gateway is cut off at 29 seconds, and the client gets a 504 while the function keeps running and billing. Configure the function timeout below the gateway's:

aws lambda update-function-configuration --function-name orders --timeout 25

For genuinely long work, return 202 immediately and do the work asynchronously, put a message on SQS or start a Step Functions execution, then let the client poll or receive a webhook.

Cold starts

A first invocation pays initialisation cost: container start, runtime boot, then your module-level code. Three things help, in order of effect:

Move work to module scope. Clients created inside the handler are rebuilt on every invocation:

import boto3
ddb = boto3.resource("dynamodb")          # once per container
table = ddb.Table("Orders")

def handler(event, context):
    return table.get_item(Key={...})      # reuses the connection

Keep the package small. A 200 MB deployment package initialises slower than a 5 MB one. Bundle only what you import.

Provisioned concurrency removes cold starts for a fixed hourly cost:

aws lambda put-provisioned-concurrency-config \
  --function-name orders --qualifier prod --provisioned-concurrent-executions 5

It requires a version or alias, not $LATEST. Use it for latency-sensitive user-facing paths and not for everything, since you pay whether or not the capacity is used.

Debugging a 502

In order:

# 1. Did the function even run?
aws logs tail /aws/lambda/orders --since 10m --follow

# 2. What did the gateway see?
aws logs filter-log-events --log-group-name /aws/apigw/orders \
  --filter-pattern '{ $.integrationError = "*" }' --max-items 5

No Lambda log at all means permissions. A log showing a successful run means the response shape is wrong. A log showing an exception means the code failed, and the 502 is hiding it, which is why you should catch exceptions and return a real status code.

Common errors

{"message":"Internal server error"}

502. Response shape, unhandled exception, or missing permission, in that order of likelihood.

Execution failed due to configuration error: Invalid permissions on Lambda function

Missing lambda:add-permission for apigateway.amazonaws.com.

Endpoint request timed out

504 at 29 seconds. Lower the function timeout and make long work asynchronous.

Works in curl, CORS error in the browser. Error responses are missing CORS headers.

Quick reference

---

Key takeaways

• body must be a JSON string. Returning an object is the top cause of 502s.
• statusCode must be an integer, and under payload 2.0 its absence silently becomes a 200.
• An unhandled exception is a 502 that hides your error. Catch and return real codes.
• Payload 1.0 and 2.0 have different event shapes. Pin the version explicitly.
• API Gateway needs a resource-based permission on the function: console adds it, CLI does not.
• Return CORS headers on error responses too, or failures appear as CORS problems.
• 29 seconds is a hard ceiling. Set the function timeout below it and offload long work.
• Initialise clients at module scope: the cheapest cold-start improvement available.

---

Check your understanding

---

Frequently Asked Questions

Why does API Gateway return "Internal server error" when my Lambda works?

That message is a 502, and it usually means the response shape is wrong rather than the code failing. The three causes, in order: body returned as an object instead of a JSON string, a missing or non-integer statusCode, and an unhandled exception, which API Gateway converts to a 502 rather than surfacing your error. Check the function's CloudWatch log first: a successful run with a 502 at the client points at the response shape, and no log at all points at a missing invoke permission.

What is the difference between payload format 1.0 and 2.0?

They are different event and response contracts. Under 1.0 the method is event["httpMethod"] and the path is event["path"]; under 2.0 they move to event["requestContext"]["http"]["method"] and event["rawPath"], cookies get their own array, and statusCode becomes optional in the response. HTTP APIs default to 2.0 and REST APIs are always 1.0, so code moved between them breaks. Pin --payload-format-version explicitly, and always set statusCode, under 2.0 its absence silently produces a 200.

Why does my integration fail with "Invalid permissions on Lambda function"?

API Gateway needs a resource-based policy on the function allowing apigateway.amazonaws.com to call lambda:InvokeFunction. Creating the integration in the console adds it for you; creating it via CLI, Terraform or CloudFormation does not, which is why this appears in infrastructure-as-code setups and not in console experiments. Add it with aws lambda add-permission, scoping --source-arn to your API so other accounts cannot invoke the function.

How do I fix CORS with API Gateway and Lambda?

Two halves. Configure CORS on the API so the gateway answers OPTIONS preflight requests itself. Then have your function return Access-Control-Allow-Origin on every response, including errors, with a proxy integration the gateway does not add headers to your responses. The tell-tale symptom of getting only the first half right is an API that works in curl and shows CORS errors in the browser whenever something fails, because the error response lacks the headers.

What is the maximum timeout for API Gateway with Lambda?

29 seconds, and it is a hard limit on REST APIs that no support request will raise. Lambda itself allows up to 15 minutes, so a long-running function behind a gateway is cut off at 29 seconds while continuing to run and bill. Set the function timeout below the gateway's, 25 seconds is a sensible ceiling, and for genuinely long work, return 202 immediately after queueing to SQS or starting a Step Functions execution, then let the client poll or receive a callback.

How do I reduce Lambda cold starts behind API Gateway?

In order of effect: initialise SDK clients and database connections at module scope so they persist across invocations in the same container; keep the deployment package small, since initialisation time scales with what has to be loaded; and use provisioned concurrency for latency-sensitive paths, which eliminates cold starts entirely for a fixed hourly cost and requires a version or alias rather than $LATEST. Choosing a faster-starting runtime also helps, though rarely enough to justify a rewrite on its own.