I am looking for someone to guide me through a setup on AWS. The process would be that I would ask questions by mail and the freelancer would answer them and will charge by the hour (15 minute chunks). I have 2 steps below. I want to get step 1 first and then maybe step 2 and further development after that.
I want to set up Elasticsearch on AWS and run a python script to download and index all fields in the full available twitter stream. I have run the script successfully on docker but not with all the fields (it is similar to the script here: [login to view URL]). As a second step, I may want to store the tweets in redis in S3. I am having trouble figuring out how to work with AWS. I installed awscli. I want someone to guide me through getting at least step 1 by answering my questions. You should have experience with ES on AWS.
I have Amazon Elasticsearch running. I know it is running because when I go to
search-wbcelastic-lmmjhaf.......[login to view URL]
I can see
{
"status" : 200,
"name" : "Arkon",
"cluster_name" : "331114504081:wbcelastic",
"version" : {
"number" : "1.5.2",
"build_hash" : "62ff9868b4c8a0c45860bebb259e21980778ab1c",
"build_timestamp" : "2015-04-27T09:21:06Z",
"build_snapshot" : false,
"lucene_version" : "4.10.4"
},
"tagline" : "You Know, for Search"
}
But when I do this command at prompt in windows powershell
PS C:\Users\hpBill> aws ec2 describe-instances
PS C:\Users\hpBill>
Does that mean I don't have an instance running?
I am working on Windows 10 as my home computer
How can I upload a python script called [login to view URL] to run in the virtual machine. I normally use a windows program called WinSCP to upload files to a virtual machine. Also where do I upload it to? Typically I upload it to usr/bin but I don't know how to access that. And then what is the cli command to run the script.
My daily job is managing Elasticsearch clusters in an enterprise environment. These clusters are all hosted in AWS. I also worked for Amazon for 3 years, so I know a lot about AWS and how it works.