Amazon Marketing Stream V2: What Changes If You Built on V1

Amazon made Stream V2 generally available on 30 September 2026 (release notes). Its September 2026 deprecation notice gives V1 a hard shutoff of 30 June 2027, the same day v3 reporting stops (deprecations), and Amazon's GitHub folder of Marketing Stream templates now calls every V1 resource "outdated" (amzn/ads-advanced-tools-docs).

I've built and run V1 pipelines for clients: Firehose into S3, then BigQuery and Looker Studio, hourly. I haven't cut one over to V2 yet, so where this post predicts a break, it's Amazon's documentation read against V1 code I know well. Everything here is as of October 2026.

Sources: Amazon's V2 overview, data delivery and ads-performance pages. One thing V2 doesn't change is the AMS-versus-AMC split: it's still aggregated performance data pushed to you, not AMC's event-level clean room.

• budget-usage and the campaign datasets. Amazon's deprecation notice says entity-change and budget-consumption messages are "not carried forward in Stream v2" and points to the Campaign Management API instead. A pacing job that reads budget-usage from Stream needs a new source, not a new subscription.
• Keyword text and match type. ads-performance-v1 carries the target as adGroupTarget.id only. Amazon's V2 FAQ says to pull mutable attributes from the campaign-management APIs and treat Stream as fact data.
• Every search term. searchTerm.value is subject to reporting thresholds; terms below them are grouped into an (Other) row.
• The 7d/14d conversion columns. V1's attributed_sales_7d-style columns are gone. Amazon's release note says the standardized metrics, which V2 shares with the new Reporting API, "use the correct look-back for each campaign type automatically".

Amazon's migration guide comes down to: build V2 beside V1, validate, then retire V1. This is how I'd do it for the shape I've shipped, Firehose into S3 and on to BigQuery. An SQS consumer follows the same steps.

1. Deploy the destination from Amazon's template

   One stack per client application, not per dataset: a bucket, a notification queue with a dead-letter queue, two customer-managed KMS keys and access logging. It contains no IAM roles, so no CAPABILITY_NAMED_IAM.

   curl -sLO https://raw.githubusercontent.com/amzn/ads-advanced-tools-docs/main/amazon_marketing_stream/StreamV2_S3_CF_Template.yaml
   
   aws cloudformation create-stack --region eu-west-1 \
     --stack-name ams-v2-destination \
     --template-body file://StreamV2_S3_CF_Template.yaml \
     --parameters ParameterKey=DestinationName,ParameterValue=ams-v2-myapp \
                  ParameterKey=StreamRealm,ParameterValue=EU
   
   aws cloudformation describe-stacks --region eu-west-1 --stack-name ams-v2-destination \
     --query "Stacks[0].Outputs[?OutputKey=='StreamDestinationConfiguration'].OutputValue" \
     --output text

   That output is the bucket and queue ARNs, shaped for the subscribe body. The data bucket survives a stack delete on purpose; the template's comment says "Backfill is capped (70 days automatic, extended lookback on request), so deleted data may be unrecoverable."

2. Swap profile IDs for advertiser account IDs

   V1 subscriptions hang off a profile ID in the Amazon-Advertising-API-Scope header. V2 takes a Global Account (amzn1.ads-account.g.…) or a DSP regional account in the body. POST /adsAccounts/list returns each adsAccountId with its per-country profile IDs, so one V2 subscription can replace several V1 ones. Manager accounts aren't accepted.

3. Subscribe, and read the 207 properly

   curl 'https://advertising-api-eu.amazon.com/adsApi/v1/create/streamSubscriptions' \
     -H 'Amazon-Advertising-API-ClientId: {clientId}' \
     -H 'Authorization: Bearer {token}' \
     -H 'Content-Type: application/json' \
     --data '{
       "accessRequestedAccounts": [{"advertiserAccountId": "amzn1.ads-account.g.xxxx"}],
       "streamSubscriptions": [{
         "destination": {"s3Destination": {
           "bucketArn": "arn:aws:s3:::{bucket}",
           "notification": {"queueArn": "arn:aws:sqs:eu-west-1:{awsAccount}:{queue}"}}},
         "configuration": {"dataset": {"datasetId": "ads-performance-v1"}}
       }]
     }'

   A per-item failure still comes back as HTTP 207. Success is an entry under success with "status": "ACTIVE"; failure is an entry under errors, such as BAD_REQUEST with A subscription already exists for this dataset, destination, and account combination. Call the endpoint for the bucket's geo: Amazon rejects an NA call with an EU destination.

4. Replace the loader, not just the path

   Read the SQS notifications, each of which lists the exact Parquet files to load, and land every record in a staging table. This is the shape I'd start from; it hasn't run against a live V2 subscription yet, so test it on one delivery first.

   import io, json
   import boto3, pyarrow.parquet as pq
   from google.cloud import bigquery
   
   REGION = "eu-west-1"
   QUEUE_URL = "https://sqs.eu-west-1.amazonaws.com/<acct>/<name>-<acct>-eu-west-1-queue"
   STAGING = "my_project.ams_v2.ads_performance_staging"
   sqs, s3 = boto3.client("sqs", region_name=REGION), boto3.client("s3", region_name=REGION)
   bq = bigquery.Client()
   
   def load(uri):
       bucket, key = uri.removeprefix("s3://").split("/", 1)
       table = pq.read_table(io.BytesIO(s3.get_object(Bucket=bucket, Key=key)["Body"].read()))
       # BigQuery won't load Parquet columns with "." in the name: campaign.id -> campaign__id
       table = table.rename_columns([c.replace(".", "__") for c in table.column_names])
       buf = io.BytesIO(); pq.write_table(table, buf); buf.seek(0)
       bq.load_table_from_file(buf, STAGING, job_config=bigquery.LoadJobConfig(
           source_format="PARQUET", write_disposition="WRITE_APPEND",
           schema_update_options=["ALLOW_FIELD_ADDITION", "ALLOW_FIELD_RELAXATION"])).result()
   
   while True:
       for msg in sqs.receive_message(QueueUrl=QUEUE_URL, MaxNumberOfMessages=10,
                                      WaitTimeSeconds=20).get("Messages", []):
           note = json.loads(msg["Body"])
           for uri in note["payload"]["batchedS3DataUris"]["s3Uris"]:
               load(uri)
           sqs.delete_message(QueueUrl=QUEUE_URL, ReceiptHandle=msg["ReceiptHandle"])

   A message is deleted only after its files load, so a crash means a redelivery, which the next step makes harmless. Amazon targets about 256 MB per file: size the worker's memory for that, and set the queue's visibility timeout above your slowest load.

5. Keep the latest version of each record, never the sum

   This is the rule that flips. A V1 consumer adds up deltas per hour after deduping on idempotency_id. A V2 consumer keeps the row with the highest streamBatch.version per record key (every dimension field) and drops rows whose metrics are all null, which is how Amazon sends a delete. Building the view from the table's schema survives new columns, and keying on the dimensions as JSON copes with the many that are null on Sponsored Products rows:

   def refresh_current_view(current="my_project.ams_v2.ads_performance_current"):
       cols = [f.name for f in bq.get_table(STAGING).schema]
       meta = ("metric__", "streamBatch__", "streamDataset__", "streamSubscription__")
       dims = [c for c in cols if not c.startswith(meta)]
       metrics = [c for c in cols if c.startswith("metric__")]
       key = "TO_JSON_STRING(STRUCT(" + ", ".join(f"`{c}`" for c in dims) + "))"
       all_null = " AND ".join(f"`{c}` IS NULL" for c in metrics)
       bq.query(f'''CREATE OR REPLACE VIEW `{current}` AS
         SELECT * EXCEPT(rn) FROM (
           SELECT *, ROW_NUMBER() OVER (PARTITION BY {key}
                                        ORDER BY streamBatch__version DESC) AS rn
           FROM `{STAGING}`)
         WHERE rn = 1 AND NOT ({all_null})''').result()

   A view is fine for one account. For many, materialise it on a schedule into a date-partitioned table; a view re-ranks the whole staging table on every query.

6. Map the columns your dashboards read

   V1 field	V2 field (. becomes __ after the rename)
   advertiser_id, marketplace_id	advertiserAccount.id; the legacy profile ID is advertiserAccountAlternateProfileId.value
   campaign_id, ad_group_id, ad_id	campaign.id, adGroup.id, ad.id
   keyword_id	adGroupTarget.id (confirm against your stored IDs in the parallel run)
   keyword_text, match_type	Not in the dataset; join your campaign data
   placement	placementClassification.value
   time_window_start	date.value + hour.value, with timeZone.value
   impressions, clicks	metric.impressions, metric.clicks
   cost (a double) + currency	metric.totalCost (decimal 18,8) + budgetCurrency.value
   attributed_sales_*, attributed_conversions_*, attributed_units_ordered_*	metric.sales, metric.purchases, metric.unitsSold
   idempotency_id	None: the record key plus streamBatch.version

7. Run both, and reconcile before you trust V2

   Amazon's own step here is "Validate against Stream v1" with both running. I'd compare in this order:

   • Traffic first: impressions, clicks and cost per campaign over a full week inside the overlap, V1 summed against V2's latest version. Traffic settles within a couple of days in my V1 data; conversions restate for up to 70 days.
   • Then the clock. In my V1 data, time_window_start was UTC for one marketplace and a local offset for another. V2 has date.value, hour.value and timeZone.value instead. Line up a few campaigns' hourly impressions against V1 and find the offset that matches before joining V2 hours to anything, Vendor RRA included.
   • Conversions last, against the V1 look-back window that matches each ad product.

   Set the tolerance before you look at the numbers, and write down every difference you accept.

8. Cut over at a boundary, then retire V1

   Don't union V1 and V2 rows for the same hours. Amazon gives the same warning about mixing two V2 dataset versions in one table: both carry the same periods, so combining them double-counts. Point the reporting view at V1 before a cutover timestamp and at V2 from it onward. Once reconciliation passes, you can move the boundary back across V2's 70 days of history and keep V1 only for older data. Then archive the V1 subscriptions and keep their S3 objects. Amazon is explicit about the order: keep V1 running until V2 is validated.

   One catch for hourly work: Amazon's FAQ says history "may arrive at a coarser grain (for example, daily) even when current deliveries for that dataset are hourly". Put the hourly boundary where V2's hourly rows actually start.

None of these come from a V2 cutover I've done. They're where Amazon's and Google's docs, read against V1 code I've shipped, say a working pipeline will break quietly.

• Summing V2 rows. Right for V1 deltas, wrong for V2 totals: a restated hour is counted again every time it's restated. Nothing errors; the numbers just drift up.
• Pointing the BigQuery transfer at the bucket. The S3 transfer from my V1 BigQuery guide accepts Parquet, but Google's rule is blunt: "You cannot load Parquet files containing columns that have a period (.) in the column name." Amazon's docs show every V2 column named that way. The objects are also KMS-encrypted, so the reader needs kms:Decrypt on the template's data key as well as s3:GetObject, and the queue consumer needs it on the queue key.
• Deduping by file. Amazon says a redelivery can arrive under a different S3 key and streamDeliveryId. Skip "seen" files and you silently drop restatements.
• Checking for HTTP 200. Create, retrieve and delete answer 207, so a V1 client that checks the status code reports success on a failed subscription.
• Trusting ACTIVE. Amazon's V2 docs now say it outright: Stream only checks that the destination exists, so a bucket policy missing s3:PutObject leaves you ACTIVE with no data and no error. Same V1 lesson: alarm on silence.
• Parsing streamBatch.version as a float. Amazon warns it can exceed what a double holds exactly. Python and BigQuery's INT64 are fine; a JavaScript consumer isn't.
• One bucket per client. An application can register five destinations in total, so an agency needs one shared destination filtered on advertiserAccount.id, not a bucket per brand.

Amazon still charges nothing for Marketing Stream. Its V2 FAQ lists the AWS drivers: S3 storage for the Parquet you keep, SQS for notifications, Athena or Glue if you query in place, and KMS for a customer-managed key. The template creates two keys; AWS prices each at $1 a month, plus $1 a month for each of the first two rotations, which the template turns on. The Firehose line in my V1 cost breakdown goes away.

30 June 2027 is the deadline, but Amazon also says some V1 functionality "may be limited" before then, so I wouldn't run V1 into its last month. Starting fresh? Skip V1. If V1 feeds anything that writes to live campaigns, such as a dayparting worker, move the dashboards first and the automation last, once you trust V2's hourly numbers.

When does Amazon Marketing Stream V1 stop working?

Amazon's deprecation notice sets a hard shutoff of 30 June 2027 for Stream V1 and all its subscriptions, including reporting, messaging and recommendation data delivered to Firehose and SQS. Existing subscriptions keep delivering until then.

Does Amazon Marketing Stream V2 backfill historical data?

Yes. A new V2 subscription delivers the dataset's restatement period, 70 days for both current datasets, within about a day of activation. Amazon notes the history can arrive at a coarser grain, such as daily, than the hourly data that follows. V1 had no backfill.

Can I get more than 70 days of history from Stream V2?

Amazon's V2 FAQ says the restatement period is fixed per dataset. Comments in Amazon's V2 CloudFormation template mention "extended lookback on request", but the developer guide doesn't say how to request it, so ask Amazon Ads API support before promising anyone more than 70 days.

Can I keep using Firehose with Marketing Stream V2?

No. V2 delivers only to an S3 bucket paired with an SQS queue in your own AWS account, in a region allowed for your geo. Firehose and SQS-payload delivery are V1 only.

What replaces sp-traffic and sp-conversion in Stream V2?

One dataset, ads-performance-v1, replaces the Sponsored Products, Sponsored Brands, Sponsored Display and Amazon DSP performance datasets. Filter on adProduct.value for the ad product you want.

Can BigQuery load Marketing Stream V2 Parquet files directly?

Not as delivered. Amazon's documentation shows V2 columns named with periods, such as campaign.id, and Google's documentation says BigQuery cannot load Parquet files with a period in a column name. Rename the columns first, for example with PyArrow.

• Amazon: V2 FAQ · Process your data · V2 CloudFormation template
• Google: Loading Parquet data into BigQuery
• Databaaba, V1: Firehose setup · the subscriberRoleArn fix · S3 to BigQuery · AWS costs