Duplicate line remover software - 4


I need a duplicate line remover desktop software.


I have this text file -






After I run the software, I will get -





I have a software to do this alredy (2 of them), but 1 can only take 1 file at a time but work very fast, the other can take many files at a time but work very slow, and I want a software that take many files at a time and work very fast.

I have 2300 txt files, each is 100 mb, I want to remove duplicate lines in all these files.


I will open the software, click browse, select folders (I have 83 folders, each folder have 27 txt files).I will click delete duplicate, and software will delete duplicate on all [url removed, login to view] will treat each file seperately when looking for [url removed, login to view] will be multi-threaded, and it will dedicate 1 thread per file, if total there is 2300 files, software will run with 2300 threads, and software will work fast also.

Software will use full potential of my windows server resources - ram and cpu processor, I have 2.93 ghz dual quad core and 8 gb ram, I want the software to use all the resources and clean up the file quickly.

My server is 64 bit and windows 2003.

I have alredy posted a project but reposting it as the guy couldn't make it.

I want it done in 1-2 hours, I don't want to waste time.

I hope I have describe exactly what I wanted, I will give u 2-3 files (zip it so it becomes small) so u can test.

I have low budget, will choose the one who can charge low.

Happy bidding.


Evner: .NET, AJAX, Delphi, Java, Python

Se mere: delete duplicate lines, u line, thread up, small project in python, make line, file processor, duplicate line remover, multi-threaded, 100 gb mb, aaa a, threaded, small python project, Remover, multi core, line work, java thread, java core, desktop software, delete duplicate, core java project, click software, bidding server java, reposting software, use java python, windows core

Om arbejdsgiveren:
( 60 bedømmelser ) bangkok, Thailand

Projekt ID: #996599

Tildelt til:


Hello sir. Please check your PM

$60 USD på 1 dag
(36 bedømmelser)

2 freelancere byder i gennemsnit $130 på dette job


Please check your PM.

$200 USD in 15 dage
(61 bedømmelser)