I have a Talend Open Studio for Data Integration job setup that needs to be completed and deloyed. The architecture of the job can be changed based on your skill-level of improving the process.
The goal is to migrate data via the Talend ETL job from PostgreSQL to SQL Server every 10-15 minutes. Talend can develop the shell script or batch file automatically after exporting the completed project, so the user will need to either have an alternate route to doing this or have Talend Open Studio installed already and be knowledgeable on the application.
We have around 80 tables that need to be initially pulled over (in full) and then a WHERE condition or filter added to just pull changes to the data (based on created_date or updated_date within last 24 hours, as an example).
The table updates all need to start at the same time and be optimized for speed and efficiency. I want this parallel style of processing through the table updates, as opposed to sequential, ordered updates, because we need this job to go as quick as possible and a small quick job not have to wait on a predecessor long-running job (could be 4-5 minutes long).
I can export and send the project we have now which has the PostgreSQL and JDBC(SQL Server) connections already setup, and a set of some of the tables already.
I require the job to be updated via a properties file to control the connection details so that we can replicate this job over and over for multiple databases just be changing the connection (not duplicating the jobs over and over again every time we create a new database).
I need this done in about 4-5 days if possible and can start ASAP.
6 freelancer đang chào giá trung bình $242 cho công việc này
Hi we have experience developer working under us and we can easily manage your work. Please fine the attached resume of our developer for your reference Let us know if you would like to proceed with more details