edufour/MoodleScraper
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Add contact info to gather archive data

Browse Source
This commit is contained in:
edufour
2024-01-19 19:11:09 +00:00
parent 3dc76ea34c
commit a84509b9bc
1 changed files with 43 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files

85
README.md
View File

@@ -1,42 +1,43 @@
# MoodleScraper # MoodleScraper
A simple tool to download as many files from moodle as possible. A simple tool to download as many files from moodle as possible.
This was designed for me to easily build an archive of my courses at EPFL. It is still in development and was not built for other moodles. This was designed for me to easily build an archive of my courses at EPFL. It is still in development and was not built for other moodles.
If used and if you don't mind, please send me the result at edouard.dufour@micro-ondes.ch for my personal archive.
## Requirements
This script was written in python 3.10.4 and the following packages: ## Requirements
This script was written in python 3.10.4 and the following packages:
| package | version |
|----------------|---------| | package | version |
| beautifulsoup4 | 4.12.2 | |----------------|---------|
| requests | 2.31.0 | | beautifulsoup4 | 4.12.2 |
| requests | 2.31.0 |
## Features
This script should be able to download all file directly linked to in moodle ressources and folders. It is not able to download files from secondary sources. ## Features
This script should be able to download all file directly linked to in moodle ressources and folders. It is not able to download files from secondary sources.
## Usage
The script is pretty simple to use. Just provide a moodle link and authentication information like this: ## Usage
``` The script is pretty simple to use. Just provide a moodle link and authentication information like this:
python main.py -u USER -p PASSWORD -d /moodle -m https://moodlearchive.epfl.ch/2019-2020/ ```
``` python main.py -u USER -p PASSWORD -d /moodle -m https://moodlearchive.epfl.ch/2019-2020/
If providing an url that doesn't end with the year, you must specify the year like this: ```
``` If providing an url that doesn't end with the year, you must specify the year like this:
python main.py -u USER -p PASSWORD -d /moodle -m https://moodle.epfl.ch/ -y 2023-2024 ```
``` python main.py -u USER -p PASSWORD -d /moodle -m https://moodle.epfl.ch/ -y 2023-2024
For speed or storage reasons you can tell the script to not download a specific kind of file: ```
``` For speed or storage reasons you can tell the script to not download a specific kind of file:
python main.py -u USER -p PASSWORD -d /moodle -m https://moodlearchive.epfl.ch/2019-2020/ -i mp4 -i mov ```
``` python main.py -u USER -p PASSWORD -d /moodle -m https://moodlearchive.epfl.ch/2019-2020/ -i mp4 -i mov
```
| argument | short | example |
|--------------------|-------|-------------------------| | argument | short | example |
| --username | -u | username | |--------------------|-------|-------------------------|
| --password | -p | superstrongpassword | | --username | -u | username |
| --moodle_url | -m | https://moodle.epfl.ch/ | | --password | -p | superstrongpassword |
| --academic_year | -y | 2023-2024 | | --moodle_url | -m | https://moodle.epfl.ch/ |
| --storage_dir | -d | /moodle | | --academic_year | -y | 2023-2024 |
| --ignore_extension | -i | mp4 | | --storage_dir | -d | /moodle |
| --ignore_extension | -i | mp4 |
## Planned
- [ ] Download submission files ## Planned
- [ ] Create url files for secondary urls - [ ] Download submission files
- - [ ] Create url files for secondary urls
-