Description:
AWS Lambdas have a 15 minute time limit. This means any data pull that requires the query to run more than 15 minutes cannot be done. There are work-arounds where a big query can be broken down to smaller queries so that it takes 15 minutes to execute. However this can be a time consuming process and limits the scenarios where the data connector can be used.
A better approach for long running jobs is to spin up a virtual machine on demand which is not restricted by the 15 minute time limit.
Use Case:
The ability to run an extraction query for more than 15 minutes will be beneficial for customers that need to send a lot of data to Visier. Currently to send a substantial amount of data to Visier via a data connector requires breaking up the data into multiple pieces, and sending them over in batches.
Mockups/Samples (if applicable):
NA
Additional Context: