Webpublisher - Needs a Disk space check and/or max file size limit
Recently we had an artist try and upload a 50GB render, it filled up our ayon server and crashed it. A general disk space check for Webpublisher ( and also ayon in general ) seems like a necessity when youre potentially working with bigger transfers.
Log in to comment and vote
Comments4
Linked to Periodic Staging Directory Cleanup
Mustafa Zaky Jafar
When I saw this request, it reminded me with Notification to Slack setting in Slack addon.
which is a maximum file size.
so, I wonder what you looking for is the same max file size? or max size of the patch?
as example, I may not be able to upload 50GB render but I can upload 100x500MB render.
Dieuwer Feldbrugge
Hey hey, well I guess thats also a good addition (I’d put the cap then on the total size of the upload regardless of files. But what I’m aiming at is:
1: Webpublisher should check the available disk space on the server and see if a pending upload will have enough space so it does not crash the ayon server by uploading a file. In general I think it would be good if Ayon itself does this too, and notifies on the dash managers/devs if the server is running out of space before it crashes hard
2: Webpublisher’s download folder should automatically clear out files it has moved, currently it keeps all uploads in that folder so it just accumulates uploads, leading to a crash of our server. I have solved it now with a cron job that removed files older than a day, but I think it makes more sense to have this handled by webpublisher itself
3: Your thing you mentioned would be a nice to have, but its not something that will fix the failure we had before, point 1 would, and point 2 would help delay the failure instead of make it inevitable, and point 3 is a nice finesse on top;.
Currently, with the way webpublisher is setup any user can willingly crash any ayon server if they’ve got a good enough internet connection and or time.
Mustafa Zaky Jafar
For no. 2, there’s a setting for it in web publisher
Retention time, setting it to 1 should give the same result as your cron job.Dieuwer Feldbrugge
Good point! I took mine down to hours with the cron job last night since even a day is long. But good shout. It still does not block some one from crashing the ayon server.