Skip to content

Commit 408b84c

Browse files
committed
Delete ready_executions by PK instead of job_id when claiming
This is another take on #229, that tries to solve a deadlock like this: ``` *** (1) TRANSACTION: TRANSACTION 5223, ACTIVE 0 sec inserting mysql tables in use 1, locked 1 LOCK WAIT 5 lock struct(s), heap size 1128, 3 row lock(s), undo log entries 2 MySQL thread id 172, OS thread handle 281471652687808, query id 11099 192.168.0.5 root update INSERT INTO `solid_queue_ready_executions` (`job_id`, `queue_name`, `priority`, `created_at`) VALUES (469, 'default', 0, '2024-05-21 01:15:11.201125') *** (1) HOLDS THE LOCK(S): RECORD LOCKS space id 12 page no 4 n bits 264 index PRIMARY of table `handson`.`solid_queue_ready_executions` trx id 5223 lock_mode X locks rec but not gap Record lock, heap no 144 PHYSICAL RECORD: n_fields 7; compact format; info bits 0 ... *** (1) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 12 page no 6 n bits 264 index index_solid_queue_poll_all of table `handson`.`solid_queue_ready_executions` trx id 5223 lock_mode X insert intention waiting Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0 0: len 8; hex 73757072656d756d; asc supremum;; ... *** (2) TRANSACTION: TRANSACTION 5227, ACTIVE 0 sec fetching rows mysql tables in use 1, locked 1 LOCK WAIT 10 lock struct(s), heap size 1128, 23 row lock(s), undo log entries 10 MySQL thread id 177, OS thread handle 281471649517504, query id 11103 192.168.0.4 root updating DELETE FROM `solid_queue_ready_executions` WHERE `solid_queue_ready_executions`.`job_id` IN (464, 465, 466, 467, 468) *** (2) HOLDS THE LOCK(S): RECORD LOCKS space id 12 page no 6 n bits 264 index index_solid_queue_poll_all of table `handson`.`solid_queue_ready_executions` trx id 5227 lock_mode X Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0 ... *** (2) WAITING FOR THIS LOCK TO BE GRANTED: RECORD LOCKS space id 12 page no 4 n bits 264 index PRIMARY of table `handson`.`solid_queue_ready_executions` trx id 5227 lock_mode X waiting Record lock, heap no 144 PHYSICAL RECORD: n_fields 7; compact format; info bits 0 ```
1 parent 7e0cecc commit 408b84c

File tree

1 file changed

+8
-7
lines changed

1 file changed

+8
-7
lines changed

app/models/solid_queue/ready_execution.rb

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -24,20 +24,21 @@ def select_and_lock(queue_relation, process_id, limit)
2424
return [] if limit <= 0
2525

2626
transaction do
27-
job_ids = select_candidates(queue_relation, limit)
28-
lock_candidates(job_ids, process_id)
27+
candidates = select_candidates(queue_relation, limit)
28+
lock_candidates(candidates, process_id)
2929
end
3030
end
3131

3232
def select_candidates(queue_relation, limit)
33-
queue_relation.ordered.limit(limit).non_blocking_lock.pluck(:job_id)
33+
queue_relation.ordered.limit(limit).non_blocking_lock.select(:id, :job_id)
3434
end
3535

36-
def lock_candidates(job_ids, process_id)
37-
return [] if job_ids.none?
36+
def lock_candidates(executions, process_id)
37+
return [] if executions.none?
3838

39-
SolidQueue::ClaimedExecution.claiming(job_ids, process_id) do |claimed|
40-
where(job_id: claimed.pluck(:job_id)).delete_all
39+
SolidQueue::ClaimedExecution.claiming(executions.map(&:job_id), process_id) do |claimed|
40+
ids_to_delete = executions.index_by(&:job_id).values_at(*claimed.map(&:job_id)).map(&:id)
41+
where(id: ids_to_delete).delete_all
4142
end
4243
end
4344

0 commit comments

Comments
 (0)