Our Data Science team takes and processes the raw data feeds... so basically a lot of hoops... Not sure if there is an easier way, or if our DS team just doesn't have time to investigate new methods....
In our case, the Raw Data feeds are automated to send to an S3 bucket, then our DS team has an automated process that picks up the files and places them in a staging DB, then they have ETL processes that take and process the staging data into Snowflake.
Not sure if that helps, or if those are the same hoops you are jumping through.
