Atomic job store transactions #3993
Assignees
Labels
comp/datastore
Related to job and execution store
Comments
This was referenced May 13, 2024
wdbaruni
added a commit
that referenced
this issue
May 17, 2024
This PR allows executing multiple job store write operations in a single
atomic transaction. This done by optionally providing a transactional
context to the operation, which will prevent automatic commit after each
operation and delegates that control to the caller.
### Usage Examples
#### A transaction per operation
This example is for the same behaviour we have today where each
operation is committed automatically, which will continue to work
```
ctx := context.Background()
var err error
// The job will be updated immediately if no error was returned
if err = s.store.UpdateJobState(ctx, jobstore.UpdateJobStateRequest{}); err != nil {
return err
}
// The execution will be updated immediately if no error was returned
if err = s.store.UpdateExecution(ctx, jobstore.UpdateExecutionRequest{}); err != nil {
return err
}
```
#### A single transaction for both operations
This is an example of the new behaviour where we can group multiple
operations in a single atomic operation
```
var err error
txCtx, err := s.store.BeginTx(context.Background())
if err != nil {
return err
}
// make sure we rollback the transaction if there is an error
defer func() {
if err != nil {
_ = txCtx.Rollback()
}
}()
// updating the job won't be committed yet
// Note we are passing the transaction context txCtx instead of a regular context
if err = s.store.UpdateJobState(txCtx, jobstore.UpdateJobStateRequest{}); err != nil {
return err
}
// updating the execution won't be committed yet
if err = s.store.UpdateExecution(txCtx, jobstore.UpdateExecutionRequest{}); err != nil {
return err
}
// commit both operations
return txCtx.Commit()
```
Closes [3993](#3993)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Today we run job store operations in their isolation transaction, which prevents us from doing atomic transactions that span multiple operations, such as handling a single scheduler plan that can involve updating the job state, updating the state of multiple executions, enqueueing delayed evaluations and adding history events.
We need way to start a single transaction and have all these operations to be committed atomically or rolled-back
The text was updated successfully, but these errors were encountered: