#16 review commands and resync the server cron with the repo

But first, also review some of the crontab-sourse.json. At this moment the gungre.ch and lib.reviews both have pending fixes, and probably more optimization.

Also, need a note about how the gungre.ch has some manually created directories. Possibly, could breakup the cron into separate jobs for each problem directory.

Also, either remove -P or add -nH to avoid the double directory references.

Also, try out -m for mirroring.

The "Not a directory" issue with /dragora seems to be an old (and probably unfixed) bug.

Good find, zPlus. I'll sign on to savannah later and include the details for this particular case.

How fun if I have enough time find a fix and to submit a patch to wget. =)

wget --mirror --timestamping --convert-links --html-extension --force-directories --no-host-directories --directory-prefix=./gungre.ch http://gungre.ch/

this seems to (almost) work with these issues:

• a bunch of .html files is created, for example dragora.html and videos.html
• .1, .2, .n files seem to be created if the server doesn't return a Last-modified header

How fun if I have enough time find a fix and to submit a patch to wget

that would be cool :P

suggestion from selk

wget has -X to exclude directories
redundacy level [probably] comes from --directory-prefix=./gungre.ch
--mirror already implies --timestamping
--wait 1
-t 3 or -t 5

And perhaps update the command for dragora mirror from cyphy.blue:

<mprodrigues> vaeringjar: I use a rsync command frusen told me to sync my mirror [16:30]

rsync -aviz --delete-delay 
  rsync://rsync.dragora.org/dragora/v2
  /usr/local/www/nginx/dragora/mirror  

<mprodrigues> likewise for v3 directory

The gungre.ch repo still has a few html files generated from the directories, but they don't seem to pile up when I run it over and over again.

The dragora repo looks good. I will check this after all the cronjobs finish in a couple hours.

Not sure if the gungre.ch piles up or not after all. The directory ~/gungre.ch/ looks like it still might, but ~/gungre.ch/dragora/repo/ only has the one. I'll give it another day.