Issue with Streaming API | Community
Skip to main content
A
Antony6790
Level 3
September 13, 2024

Issue with Streaming API

  • September 13, 2024
  • 4 replies
  • 1412 views

Hi All,

Calling Streaming API and trying to ingest data into the dataset but getting below error. 

 

"The transformed message is empty. Please check the mapping set cc70ab1fe7e044da9d5883be25cb05d1 and version 0 and XDM schema _xyz.schemas.b1a161669dd4f9f9392ccf1c182868f6270a5e9eccc5cd95 and schema version 1 and try again. Message details:"

 

Using the schemaregistry API, I got the SchemaID and using it in the API call. Also in the payload I'm sending all the attributes in the same order how I added in the dataflow mapping.

 

From the error I think streaming Ingestion API is trying to use old Schema version even though Im using schemaID of latest version 1.1 or somewhere schema version is not matching.

 

Any suggestions how to resolve this error?

 

Thanks

 

 

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

4 replies

DavidRoss91
Community Advisor
DavidRoss91Community Advisor
Community Advisor
September 13, 2024

Hi @antony6790 

Could you send a copy of the payload you are sending? You can remove any sensitive info. 

    A
    Antony6790Author
    Level 3
    September 16, 2024

    Hi David,

    Below is the POSTMAN payload.

    {
    "messages": [
    {
    "header": {
    "schemaRef": {
    "id": "https://ns.adobe.com/xyz/schemas/<schemaID>",
    "contentType": "application/vnd.adobe.xed-full+json;version=1.0"
    },
    "imsOrgId": "xyz",
    "datasetId": "xyz",
    "source": {
    "name": "xyz"
    }
    },
    "body": {
    "xdmMeta": {
    "schemaRef": {
    "id": "https://ns.adobe.com/xyz/schemas/<schemaID>",
    "contentType": "application/vnd.adobe.xed-full+json;version=1.0"
    }
    },
    "xdmEntity": {
    "state": "ABC",
    "address": "Testing4",
    "owner_id": 6,
    "id": "A6794",
    "address2": "Schema new",
    "zip": "T1J 3Z6",
    "name": "Testing4",
    "city": "Testing4",
    "pfs_id": 1

    }
    }
    }
    ]
    }

      brekrut
      Adobe Employee
      brekrutAdobe Employee
      Adobe Employee
      September 16, 2024

      Hello @antony6790, 

       

      Your version number in the schema should always match 1 when sending in a streaming /POST to a data collection.

       

      You can use the following as reference for the body.

      { "header": { "schemaRef": { "id": "<schemaID>>", "contentType": "application/vnd.adobe.xed-full+json;version=1.0" }, "imsOrgId": "<<IMSORG>>", "datasetId": "<<dataSetID>>", "source": { "name": "<<dataflowName>> " } }, "body": { "xdmMeta": { "schemaRef": { "id": "<schemaID>>", "contentType": "application/vnd.adobe.xed-full+json;version=1.0" } }, "xdmEntity": { "_<<tenant>>": { } } } }

       

       

        A
        Antony6790Author
        Level 3
        September 16, 2024

        Hi Brekut,

        Yes, I'm using 1.0 as version. However in the schemaregistry I see the version for this schema as 1.1 and tried this version also, but still getting same error.

         

        Thanks.

          DavidRoss91
          Community Advisor
          DavidRoss91Community Advisor
          Community Advisor
          September 16, 2024

          @antony6790 might not matter but I see "messages" at the top of your payload. Can you try starting with just the following below?


          {
          "header": {

            RiteshY18
            Community Advisor
            RiteshY18Community Advisor
            Community Advisor
            September 17, 2024

            @antony6790 it seems to be an error due to transformation and mapping during the ingestion process, you have payload but during the process, throwing error may be one of the below 

            - transformation logic if any 

            -  schema mapping 

            -  data field type/ required field 

             

              kautuk_sahni
              Community Manager
              kautuk_sahniCommunity Manager
              Community Manager
              October 16, 2024

              @antony6790 Did you find the suggestions helpful? Please let us know if you require more information. Otherwise, please mark the answer as correct for posterity. If you've discovered a solution yourself, we would appreciate it if you could share it with the community. Thank you!

              Kautuk Sahni
                Terms & ConditionsAccessibility statement

                Loading footer...