Đã Hủy

Building a Prototype Environment for Talend and HDP + Crawler - repost

We are looking for a company or group or individual who has experience in Talend and HDP. (Hortonworks Data Platform)

This project is prototype project to verify the technical issues before the real project comes.

Therefore, if the result of this project is good, we will hire you for the real project also.

If you have experience of Talend, HDP and JAVA, this project is NOT hard to perform.

Only experienced candidates will be welcomed.  

1. By user-defined keywords, extract specific data(mainly, area) of a specified website and store the data by a text file or db format on my server.

2. By a scheduled Job, inject the extracted data into HDP. And the job must be designed by Talend Open Studio.

- The process condition for the job will be defined by discussion as you design the job. 

- Create 3~5 tables on a mySQL DB for the result

- The transfer from HDP to mySQL DB is performed by Talend. 

- The design of the tables will be discussed as you reach that step.

** Reference : The configuration of the prototype system is attached.

1. First, you should install all the components on my servers by remote. (TeamViewer will be used) 

2. You should work on my servers from the beginning. 

3. All the installation steps must be transferred to my staff by following your way.( Skype will be used for this communication)

: This is the key principle condition for payment.

1. Linux Web Crawler : you should prepare a proper crawler to satisfy the needs, below.

: This crawler must be offered to us. and we can ask some additional customizing needs for the project propose. (discussion needed)

- Searching specified Web site by keywords (including the subdirectories)

- Extraction should be repeated by time setting.

- Extracting Items

a. URL

b. meta tag(title, description, keyword) 

c. plain text between to tag

d. page size

e. last modified date value

2. Talend Open Studio for Big data (free version)

- Already Installed on my machine. but, if you need, you can use user own.

- But, the result project files(Talend project files) must be offered us.

3. HDP  (free version)

- Already Installed on my machine. and you should work on it during this project.

- You can reinstall it or change the configuration of it if you need.

- All the changed and modified history must be transferred to us.

** This project will start by this Wednesday(22th/June), at 9:30am (GMT+9).

** The bid will be closed until this Tuesday(21th/June), at 11pm(GMT+9).

** The desired due date for completing this project is this Sunday, at 2pm(GMT+9)

Kỹ năng: Kĩ thuật, Hadoop, Java, MySQL

Xem thêm: technical engineering job, technical design website linux, searching java, prototype web page design, prototype reference, prototype items, prototype engineering for hire, prototype date, principle of design, looking for experienced engineering, job web crawler, Hortonworks, hire principle, free web site building, engineering building design website, desired job title, create a your website for free, create a web store for free, create a prototype, building prototype, building a website design company, web teamviewer, talend studio, remote staff, big data java

Về Bên Thuê:
( 23 nhận xét ) Hwaseong-si, Korea, Republic of

Mã Dự Án: #4540338

1 freelancer đang chào giá trung bình $1030 cho công việc này

skysap

I can start this job right now. I read your job description very carefully. Please check PMB

$1030 USD trong 10 ngày
(1 Đánh Giá)
1.0