AWS S3 text files to Relational Database

A little background - I am using AWS to manage a mountain bike timing system. Competitors tag on and off a number of different stages and the tag off time minus the tag on time is the stage racetime and the lowest cumulative time over all stages wins.

Out on the racecourse, I am using Raspberry Pi's with RFID readers which collect and store data in text files which automatically upload to the AWS S3 bucket. I currently have two files in there: Stage 1 [login to view URL] and Stage 1 Finish.txt.

I need to get these files into a database to join with a competitor registration database so I can ultimately compute racetimes in AWS, extract some analytics and allow competitors to search and compare times using their surname or racing number to find their race times.

In short, I need help converting these text files into a relational database and this process is automated so that when new items are appended to the text files that they update the database.

I think a Data Pipeline is the way to go but happy to listen to alternative recommendations to achieve my ultimate goals as described above.

The text file item/row is semi-colon delimited as follows:

[login to view URL];1s;2016/12/30;13:56:23.235;00235689

Meaning log information (not needed), 1s (Stage 1 Start), Date, Time to milliseconds and finally the RFID code.

Any help with the Data Pipeline (or another alternative) is required as soon as possible.



Evner: Amazon Web Services, Database Administration, MySQL

Se mere: translate audio files text, auduio files text, exporting ning database files, mysql, relational database video rent, psd examples files text, relational database project, transcribe sound files text, project database files mega retail store, type picture files text money, perl search files text, php mysql relational database frontend, transcribe audio files text, mysql database hold text files, create html files text files, audio files text

Om arbejdsgiveren:
( 12 bedømmelser ) Mandurah, Australia

Projekt ID: #12676953

Tildelt til:


I am highly skilled in the AWS environment. I have been working with AWS for 5 years now and run multiple production workloads across many AWS services. I have both the AWS Devops Engineer and AWS Certified Solutions Flere

$250 AUD in 3 dage
(23 bedømmelser)

18 freelancere byder i gennemsnit $172 på dette job


I have 9 years experience in JAVA, J2EE, WebServices and enough experience in AWS webservices. I am not sure if we can dine the pipeline such that it extracts te file and fills the required columns of a relational dat Flere

$166 AUD in 5 dage
(25 bedømmelser)

Hi, Firstly, Thank you for reviewing my proposal. I have gone through your requirements and came to know that you need an expert web developer to build your website. I have strong knowledge and 3+ years experience i Flere

$222 AUD in 3 dage
(14 bedømmelser)

Please come online for detail. My Skype Id is [login to view URL] I am a highly skilled senior developer with extensive development experience in system design, .NET technology stack including C#, ASP.NET, MVC, Web Api, SQL Flere

$155 AUD in 3 dage
(15 bedømmelser)

I can write a php script which can load data from text file to database. I would love ot work on something new and good thing ..

$155 AUD in 3 dage
(14 bedømmelser)

Hello there. I think lambda could be the solution for you. I can take care of this task if you wish I am an experienced developer with strong aws knowledge

$155 AUD in 3 dage
(1 bedømmelse)

Hello there i have read your description, i will recommend a web application for this, can you please contact to discuss in details? i am waiting for your pm thanks

$250 AUD in 3 dage
(7 bedømmelser)
$144 AUD in 3 dage
(3 bedømmelser)

Hi, I have over 7 years of experience as a database developer. I have completed three recent projects. All clients were 100% satisfied as you can see from their reviews. One of the projects I completed recently was Flere

$170 AUD in 3 dage
(7 bedømmelser)

sir,this is easy task for me.i will complete your work as soon as possible with out any [login to view URL] give me the chance to work with [login to view URL] for your reply. Thanks

$35 AUD på 1 dag
(5 bedømmelser)
$111 AUD in 3 dage
(8 bedømmelser)

Hi sir, We assure you the quality of service through various checks that are employed to avoid any compromise on quality of work. Let me introduce my team to you and generally what services we offer to our client Flere

$150 AUD in 8 dage
(1 bedømmelse)

i have similar experience working with S3 and mysql db.

$222 AUD in 4 dage
(2 bedømmelser)
$155 AUD in 3 dage
(0 bedømmelser)

you don't have to load your data to different database unless you have compelling reasons. Now amazon athena supports query the s3 data like sql queries. Much less expensive and less maintenance than using different da Flere

$155 AUD in 3 dage
(0 bedømmelser)

Hello! I can do the job u need. I think Python is the best solution for you as it's covering all your needs. On the other hand maybe it makes sense to use logstash to transform and send data to your database and do Flere

$222 AUD in 10 dage
(0 bedømmelser)

Have 4.5 years of experience in Java, REST, Data Structures and Algorithms, SQL and NoSQL(MongoDB, Redis), Spring, Hibernate hence I will be a good help for this project. I had recently worked with Rajhees, C.E.O, MD , Flere

$155 AUD in 3 dage
(0 bedømmelser)
$222 AUD in 5 dage
(0 bedømmelser)