k8s: convenience script for running MW jobs #829
Conversation
|
Could/should this be a part of |
|
Very possibly; in reality I just wanted my rough hack I ran locally to be represented somewhere within git. The key question I guess is where should the actual blocking on job completion logic really be. In general this is actually not a great paradigm for these distributed systems; when a job fails this leaves this script hanging on some developers laptop. Also if the developer's laptop falls asleep we lose the state etc. . In reality we probably want something more like: jobs are submitted, one for each wiki, they are however only scheduled with a given concurrency. Thinking out loud there are really two uses cases: wanting to flush a specific wiki after something specific happens or wanting to flush all wikis. Finally this job should really not exist at all since no manual operator should ever really be doing this. Instead we should resolve https://phabricator.wikimedia.org/T330389 so that there is never a backlog of jobs |
|
Shall we close this? |
No description provided.