Lukket

download github repos from search result or given repos list or given user do it by python code

to do

windows computer , python 3.7 code download github repos from search result or given repos list or given user

example

[login to view URL]

or

[login to view URL]

use simple packages to install like BeautifulSoup and selenium , teach me how to install

not limited by next functionalities , but all what can be done manually should be done

0

Prevent blocking like

agents = cycle([r"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36 Edg/[login to view URL]",

r"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0",

r"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36"])

and time delays

and etc

1

download repos to mimic manual search

2

Download all search results like: commits, code, issues, keywords/labels (for example

[login to view URL]

) - download whole repos, do not download the same repo twice

Like

Repositories

Code

Commits

Issues

DiscussionsBeta

Packages

Marketplace

Topics

Wikis

Users

Or

All for given use like

[login to view URL]

3

unzip all compressed files in repo - for all main zip algorithms all what 7zip can uncompress , all files in repo (even for many nested folders )

4

Save each language to separate folder for example ["Python", "Java", "Shell"]

5

Search for all languages or only for given set for example ["Python", "Java", "Shell"]

6

Set parameter for N best match or N most stars or N recently updated or N most commented etc

7

You put here needed functionalities

The script also supports to stop and continue the downloads from a previous left over instance

For example internet was disconnected for computer restarted

8

Log file for errors and not downloaded repos by some reason with error message for each case

If hard disk is full make it easy: just check disk size for each repo and when It less than 5 Gb , pause and ask to free dsik size and them resume?

add filtering by date . meaning to download only fresh repos , lets say after July 2019 or December 2019.

Or repos in dates range

Full functionality as in advanced search

[login to view URL]

Basic search

This search Finds repositories with…

cat stars:>100 Find cat repositories with greater than 100 stars.

user:defunkt Get all repositories from the user defunkt.

tom location:"San Francisco, CA" Find all tom users in "San Francisco, CA".

join extension:coffee Find all instances of join in code with coffee extension.

NOT cat Excludes all results containing cat.

9

Find data hidden some where, for example

[login to view URL]

[login to view URL]

[login to view URL] 330 MB

Evner: Python, Web Skrabning

Se mere: document management user application code, download sobi2 search plugin, phpbb3 check user group code, recursive function search linked list, word search puzzles python code, download google search results url list, search word file python code, download google search result, automatic download google search result, download google business search result, download google search result java, code list user control net, user developer code sphinx search, python graphical user interface code, depth first search python code

Om arbejdsgiveren:
( 5 bedømmelser ) Toronto, Canada

Projekt ID: #26923766

6 freelancere byder i gennemsnit $37 på dette job

Varun942

Dear Client! I read your description carefully and I'm very interested in your project. I am specialized in python and web scraping.i have done some projects like this.i can show demo when we connect. I am sure that Flere

$100 USD in 3 dage
(5 bedømmelser)
2.9
alexmahmudova

hi there. As a senior developer with 6 years of experience in Web Scraping, I have rich expertise and deep knowledge in almost all types of scraping tools published. But I always prefer using my own scraping tool writt Flere

$25 USD på 1 dag
(5 bedømmelser)
2.8
rkurilenko007

---@Faith moves a mountain@--- Hello! Scraping with python is easy but take some [login to view URL] know? I've read your description carefully. I think I can help with fixing your problems. I'm a Python, Laravel, WordPress, Reac Flere

$50 USD in 7 dage
(0 bedømmelser)
0.0
saadtariq329

Greetings! Since I have got a visual over your project's description, I have some follow-up questions to ask so to elicit the missing pieces of requirements. Please message me so we can proceed! I am looking forward to Flere

$20 USD in 7 dage
(0 bedømmelser)
0.0
Ragesh77

I can do this. Give me a chance show my skills. Lets discuss and get it done. I can finish the work within a day.

$10 USD på 1 dag
(1 bedømmelse)
0.0
sdr00007

Ummm, May be my passion and learning ability makes me perfect for this project, Because whenever i've work on any type of projects i can work on that for many hours an hours. I can give you guarantee to complete the pr Flere

$15 USD på 1 dag
(0 bedømmelser)
0.0