Lukket

need fast script to parse html using wget or curl

Hello,

The script should;

1- Crawl the webpage given

2- Parse all the urls in page with different regular expressions. (don't have to start with a href or http even)

for example: parse all urls with rar,zip,mp3 etc. extensions. parse all mediafire, rapidshare etc. urls.

3-It should be able to login or load cookies to login to specific webpages such as forums etc. to get the links

4-Must be fast as much as possible and stable :).

it can be shell script, perl, c etc. important part it should be fast and not use much resources. advices about platform or techics welcome.

below is an example which I can do till here, I need so many improvements

wget -q -U "Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.9.0.2) Gecko/20121223 Ubuntu/[url removed, login to view] (jaunty) Firefox/3.8" [url removed, login to view] -e robots=off -O - | tr "\t\r\n'" ' "' | grep -i -o '"\(ht\|f\)tps\?:[^"]\+\(.gif\|.apk\|.rar\|.mkv\)"' | sed -e 's/^.*"\([^"]\+\)".*$/\1/g' | uniq

thanks in advance

Færdigheder: Lidt af Hvert, Ingeniørarbejde, Linux, Shell Script

Se mere: rapidshare, wget grep html, using regular expressions, using expressions, regular expressions example, linux regular expressions, example regular expressions, fast script rar html, fast script rar, shell, welcome gif, example shell script, webpage improvements, uniq, shell script, rv, jaunty, ht, need curl, shell script perl, given apk, linux login script, curl grep, script apk, html mp3

Om arbejdsgiveren:
( 32 bedømmelser ) Istanbul, Turkey

Projekt-ID: #4072695

17 freelancere byder i gennemsnit $154 for dette job

SigmaVisual

I can help in your project, please check PMB and our ratings/reviews to get idea of our experience. Please let me know if you have any queries.

$225 USD in 5 dage
(48 bedømmelser)
6.1
ebson

I checked the code, If u only want the regular expression I can give it in less than an hour

$30 USD in 0 dage
(44 bedømmelser)
5.5
srinichal

I can deliver the project using regex

$180 USD in 5 dage
(14 bedømmelser)
4.4
kandamunlabs

Please see private message.

$250 USD in 4 dage
(11 bedømmelser)
3.7
asmodej

Hello! I would be glad to complete your project using Python. As you can see in my "Past Work" page, I have done very similar projects before (webpage scraping, automated posting). Your application will support adding Mere

$150 USD in 4 dage
(2 bedømmelser)
3.6
mlambrichs

Obviously it's cool to write a oneliner, but I don't think it's wise reading your requirements. Read my PM and see if you can stand all my insults. ;-)

$225 USD in 3 dage
(6 bedømmelser)
3.3
programer22

PHP5 standalone script Might be call from cron tab

$250 USD in 10 dage
(1 bedømmelse)
2.8
morissette

I can do this within 24 hours of bid acceptance.

$120 USD på 1 dag
(2 bedømmelser)
2.2
nithi87cool

Hi Dude, i have enough years of exp to fix your issue and rewrite the script. please see private message.

$100 USD på 1 dag
(1 bedømmelse)
2.1
coderz1

I have good exposure to Linux (wget,curl, scripting) and C scripting and I can code your problem in a maximum of 2 days and can deliver it with all the features. I can also provide future support/changes free of cost.

$40 USD in 2 dage
(1 bedømmelse)
1.2
apwaytechnology

I have worked with regex. I can solve your problem.

$200 USD in 15 dage
(0 bedømmelser)
0.0
Paddy0

Hi, I have written similar scripts to this before, so I'm fully aware of the requirements and potential issues that would arise - Although I would obviously like to communicate with you before I begin coding to ascert Mere

$225 USD in 4 dage
(0 bedømmelser)
0.0
PerlSQLMaster

Many years of Perl programming experience. Can do the job.

$150 USD in 3 dage
(0 bedømmelser)
0.0
zigler

An expert of shell/linux/regex from search engine company. I can finish this job. Please tell me details. Thanks

$100 USD in 2 dage
(0 bedømmelser)
0.0
ppan279

Hello, I am a seasoned webscrapper using Perl and have an extensive experience in shell scripting too. Perl is the easiest and most efficient technology for webscrapping jobs because of its robust regular expression s Mere

$180 USD in 5 dage
(0 bedømmelser)
0.0
tmrlvi

I have written several web scrappers, including logging in using browser cookies. With some Python work, it is possible to finish this project in 4 days.

$150 USD in 4 dage
(0 bedømmelser)
0.0
dudytz

I am a professional with 7 years of experience with web extraction and data processing.

$40 USD på 1 dag
(0 bedømmelser)
0.0