Extracting list of awarded badges on Moodle

A nice feature of Moodle is badges. Badges can be given to students as rewards for good actions, and incite them to do better and follow activities.

However, as often in Moodle, the functionalities are quite limited. One missing feature is the ability to give a badge to a list of user from an external system, e.g. using a CSV file. You can select students one by one in a list, which is cumbersome. As most as possible, for mass awarding you should try to use some “scripting” : for instance awarding badges when users finish an activity, with a certain grate.

The more problematic one I did not forsee was the inability to export badges. I wanted to give a small bonus points for students who won some badges. I thought of two leads : the moodle backups where I could extract the badges from the XML, and create a LTI activity to “steal” the badges.

The Moodle backups, even if you click on “Badges”, actually do not include badges in the backup… In 4 months this is the 4th bug I hit… After 3 confirmed bug that got no activity, I don’t even report them anymore… The Moodle community is rather slow, and internal resources are limited. The LTI 1.2 activities as far as I can tell do not have access to badges so that was a no-go as well.

So the solution was to do some web scraping. After trying multiple chrome extensions (connecting to moodle through GET/POST request would be very complicated because of OAuth), I could finally do a correct extraction with this extension : https://chrome.google.com/webstore/detail/web-scraper-free-web-scra/jnhgnonknehpejjnehehllkliplmbmhn/related?hl=en

This extension supports following links, because there’s of course no web pages allowing to see all badges per students. You must “click” on the number of students for each badge, and then follow the pagination as there’s a limit of 50 students per pages.

TLDR

Install the extension, go on the “Manage badges” pages on your Moodle course, and then press F12 to open chrome developer tools. Go to the new “Web Scraper” page and then “Create new sitemap” -> “Import sitemap”.

Here’s the “sitemap” defining how to follow links and scrape badge names, students that were awarded the badges, while supporting pagination.

Moodle 3.9:

{"_id":"moodle_badges","startUrl":["https://moodle.uclouvain.be/badges/index.php?type=2&id=XXXX"],"selectors":[{"id":"Awards","multiple":true,"parentSelectors":["_root"],"selector":".awards a","type":"SelectorLink"},{"id":"Element","multiple":true,"parentSelectors":["Pages"],"selector":"tbody tr","type":"SelectorElement"},{"id":"Name","multiple":false,"parentSelectors":["Element"],"regex":"","selector":".cell.c0 a","type":"SelectorText"},{"id":"Pages","paginationType":"clickMore","parentSelectors":["Awards","Pages"],"selector":"nav:first-of-type li a[aria-label=\"Next\"].page-link","type":"SelectorPagination"},{"id":"BadgeName","multiple":false,"parentSelectors":["Awards"],"regex":"","selector":"h1","type":"SelectorText"}]}

Moodle 4.x:

{"_id":"moodle_badges","startUrl":["https://moodle.uclouvain.be/badges/index.php?type=2&id=XXXX"],"selectors":[{"id":"Awards","multiple":true,"parentSelectors":["_root"],"selector":".awards a","type":"SelectorLink"},{"id":"Element","multiple":true,"parentSelectors":["Pages"],"selector":"tbody tr","type":"SelectorElement"},{"id":"Name","multiple":false,"parentSelectors":["Element"],"regex":"","selector":".cell.c0 a","type":"SelectorText"},{"id":"Pages","paginationType":"linkFromHref","parentSelectors":["Awards","Pages"],"selector":".mt-1:first li:last-child a","type":"SelectorPagination"},{"id":"BadgeName","multiple":false,"parentSelectors":["Awards"],"regex":"","selector":"h1","type":"SelectorText"}]}

Press Sitemap -> Scrape and voilà. After a while, you can download a CSV or an XLSX.

As a bonus, what I want is the number of badges per students, so here’s the formula to put on the student list where the Name and FirstName are separated columns:

=NB.SI(Badges!E:E;CONCAT([@Name];" ";[@FirstName]))

With mails on Moodle 4.1:

{"_id":"moodle_badges","startUrl":["https://moodle.uclouvain.be/badges/index.php?type=2&id=XXX"],"selectors":[{"id":"Awards","parentSelectors":["_root"],"type":"SelectorLink","selector":".awards a","multiple":true},{"id":"Element","parentSelectors":["Pages"],"type":"SelectorElement","selector":"tbody tr","multiple":true},{"id":"Name","parentSelectors":["Element"],"type":"SelectorLink","selector":".cell.c0 a","multiple":true,"linkType":"linkFromHref"},{"id":"Pages","parentSelectors":["Awards","Pages"],"paginationType":"linkFromHref","type":"SelectorPagination","selector":".mt-1:first li:last-child a"},{"id":"BadgeName","parentSelectors":["Awards"],"type":"SelectorText","selector":"h1","multiple":true,"regex":""},{"id":"Mail","parentSelectors":["Name"],"type":"SelectorText","selector":"dt:contains('Adresse de courriel') + dd a","multiple":true,"regex":""}]}