Amazon Marketing Stream: “Value null at destination.firehoseDestination.subscriberRoleArn”

You're subscribing a dataset to an Amazon Marketing Stream destination. You send the body every reference implementation shows, and Amazon rejects it:

HTTP 400
Value null at 'destination.firehoseDestination.subscriberRoleArn'

The confusing part is that your body doesn't contain subscriberRoleArn at all, so "null" reads like Amazon is complaining about a field you never claimed to send. It is. The subscribe body now requires three ARNs, and older client code sends two.

If you built from an existing Marketing Stream integration, or from a sample written against the earlier API shape, this is where you land. The fix is one line — but the field naming is genuinely confusable, so it's worth being precise about what each ARN is.

Two of the three differ by a single word, which is most of why this trips people:

body = {
    "dataSetId": data_set_id,
    "notes": "…",
    "destinationArn": firehose_arn,
    "destination": {
        "firehoseDestination": {
            "deliveryStreamArn":   delivery_stream_arn,
            "subscriptionRoleArn": subscription_role_arn,
            "subscriberRoleArn":   subscriber_role_arn,   # required — omitting it 400s
        }
    },
}

Read the error message literally when this happens: the path it gives you (destination.firehoseDestination.subscriberRoleArn) is exactly where the missing key belongs. Amazon's own Marketing Stream documentation is the authority on the current shape — check it rather than an older integration, because that's the whole cause of this error.

You fix the ARN, the subscription goes ACTIVE, records start landing in S3 — and your parser throws on the first object.

If compression is enabled on your Firehose delivery stream, objects arrive GZIP-compressed, with .gz keys. Reference parsers written against an uncompressed stream read plain JSON and fall over. Since whether compression is on depends on how your delivery stream was configured, this is inconsistent between two otherwise identical integrations.

Sniff rather than trust the file extension — S3 keys aren't always what you expect:

import gzip

def read_object(body: bytes) -> bytes:
    # gzip magic number
    if body[:2] == b"\x1f\x8b":
        return gzip.decompress(body)
    return body

Two lines, and it works whether or not compression is enabled — which also means it keeps working if someone changes that setting later.

The last surprise isn't a bug at all, but it generates a lot of "is this working?" energy on day one.

Marketing Stream is push-based and has no backfill. Data accrues from the moment your subscription becomes ACTIVE — there is no historical window, no request-past-data call, nothing to replay. Enrol on a Tuesday and Monday does not exist for you.

So if you need history, pull it from the Ads reporting API and stitch it to the stream at the enrolment boundary. And enrol earlier than you think you need to — every day you delay is a day of hourly granularity you can never get back. The reporting API can give you daily numbers for that period, but not the hourly detail that made you want Marketing Stream in the first place.

When a Marketing Stream subscription isn't producing data, check in this order — it's roughly the order of likelihood, and each step is cheap:

1. Is the subscription ACTIVE? Not created — active. A subscription can be created and then fail to activate because of the trust policy.
2. Does the trust policy on the subscription role name Amazon's account correctly? This is the most common activation failure.
3. Is Firehose actually delivering? Check the delivery stream's own CloudWatch metrics before suspecting Amazon. Delivery failures to S3 are visible there and have nothing to do with the Ads API.
4. Is your parser silently skipping objects? The gzip case fails loudly, but a broad except around object parsing turns it into "no data" — which looks like a subscription problem and isn't.
5. Has enough time passed? Hourly datasets deliver hourly. An hour of nothing after enrolment is normal.

What causes “Value null at destination.firehoseDestination.subscriberRoleArn”?

Your subscribe request is missing the subscriberRoleArn field. The Marketing Stream subscribe body requires three ARNs — deliveryStreamArn, subscriptionRoleArn and subscriberRoleArn — and older client code and samples send only the first two.

What is the difference between subscriptionRoleArn and subscriberRoleArn?

subscriptionRoleArn is the IAM role Amazon assumes in order to write records into your Firehose delivery stream. subscriberRoleArn identifies you as the subscriber. They are separate fields and both are required.

Why does my Marketing Stream parser fail on the S3 objects?

If compression is enabled on your Firehose delivery stream, the objects are GZIP-compressed. Detect the gzip magic bytes 0x1f8b at the start of the object and decompress rather than assuming plain JSON.

Can I backfill historical Amazon Marketing Stream data?

No. Marketing Stream is push-based and delivers data only from the point your subscription becomes active. For any period before that you need the Amazon Ads reporting API, which gives daily rather than hourly granularity.

How long after subscribing should data appear?

For hourly datasets, expect the first records within about an hour of the subscription becoming ACTIVE. If nothing arrives after that, verify the subscription status and the delivery stream's CloudWatch metrics before assuming the subscribe call was wrong.