Looking for someone who have strong Perl knowledge and unix shell script experience.
My budget for this work is $[url removed, login to view]
I attached 2 files which has complete code and table schema and sample data with 2 test records in [url removed, login to view] file
If you have strong on perl particularly using hash kind of skill it is easy for you to do
If you are a perl + shell expert you can easily figure this out.
because of our sql skills we used mysql to implement logic you can do substitutions with perl or shell anything. what ever method you like but that should speed up the processing.
i have 2 tasks to speed up. - Other than time nothing wrong with these scripts.
see complete code
we use perl + mysql to format data uses xls to csv file conversions.
currently most logic in formating and loading data into mysql taking long time, you can try avoid it and implement a
hash technique to speed it up.
to process 40,000 records 18mb input data - uses templates to prepare final records it takes about 3.5 hours.
1/2 of time is for record creating locally, remaining time is consuming for database load and export data back to local disk
this task involves substitutions sherch and replace by a certain logic again because we have sql skils we use databsae to perform this. if you use hash technique we can save lot of time.
we use csv files as input and operation involves 2 gb to 10gb data files to process.
Check project message board for exising code and sample tables and steps to execute the script