Output dataset and schedule functionalities in Query service not working

[nico9]

Hi all,

we are running into an issue when scheduling a query form AEP UI.

We have created a query that make a join between two dataset and combine the retrieved data into a new dataset using the insert into. 

If we run the query manually, the dataset get populated and everything looks fine, the problem comes when we try to schedule the same query, selecting in the output dataset the same dataset specified in in INSERT INTO statement.

So, what could be the problem of the schedule? it should just run hourly. 

Are there some limitation on the use of scheduling? 

[ChetanyaJain]

What's the error that you are seeing @nico9? What's the frequency of the schedule? How much time does it take to complete the Job (when tested manually)?

Also, when you tested manually did you try with delta update OR every time you delete & reload the data?

[nico9]

hi chetanyajain, there is no error, it looks like the schedule never start. If i manually run the query the output is returned in one minute more or less.

For the last question you made i actually don't know, maybe we miss something. I thought that just running the schedule the table is overwritten automatically.

[Anuhya-Y · Accepted solution]

@nico9 

 Is your query starting with Insert into “dataset name” when you schedule the query? If yes, remove Insert into “dataset name” from the query. As schedule query UI takes dataset name as input while creating scheduler.

If above question answer is ‘No’ , next question - is the scheduler working for any scheduled time (hourly, daily) ? if no, please reach out to Adobe to add advanced features (scheduling) of query service to the product license.

[nico9]

Hi anuhya-tad, no we have no INSERT INTO statement. the schedule runs hourly.

[Anuhya-Y]

@nico9 

Can you retrieve the runs and check the status using query service API https://developer.adobe.com/experience-platform-apis/references/query-service/#operation/listSchedul....

. check out state in the response

I just tried to run a sample query every 5 min. https://developer.adobe.com/experience-platform-apis/references/query-service/#operation/updateSched...

It ran successfully and inserted data in dataset.

{

"body": [

{

"op": "replace",

"path": "/schedule/schedule",

"value": "*/5 * * * *"

}

]

}

[AtulChavan]

Hi @nico9 ,
I suggest, you should check your license and see if 'Query Services' is meant to be used for ETL. I suspect in your case query service feature is to do analysis only hence scheduler never kick offs.