recentpopularlog in

queue

« earlier   
ferd/pobox: External buffer processes to protect against mailbox overflow in Erlang
High throughput Erlang applications often get bitten by the fact that Erlang mailboxes are unbounded and will keep accepting messages until the node runs out of memory. In most cases, this problem can be solved by imposing a rate limit on the producers, and it is recommended to explore this idea before looking at this library. When it is impossible to rate-limit the messages coming to a process and that your optimization efforts remain fruitless, you need to start shedding load by dropping messages. PO Box can help by shedding the load for you, and making sure you won't run out of memory.
erlang  highavailability  ha  queue  buffer 
2 days ago by dlkinney
Postgres Job Queues and Failure by MVCC | Hacker News
If you're using an external queue such as RabbitMQ, there's basically no way to guarantee a queue message being fired without a "two-phase" transactional approach.

Let's say your use case is creating a user and sending an email verification email. You need to:

1. Start transaction.

2. Create the user.

3. Insert a tuple into an "events to fire" table.

4. Commit.

5. Fire the event.

6. Delete the tuple.

This ensures that if the transaction fails, nothing is fired; if the transaction succeeds, we guarantee that the user now exists, and that the command to fire the event also exists. Thus, if step 5 fails, we still have the "events to fire" tuple, and we can have a cron job running every minute that picks up stragglers. (In practice, you'll need to run steps 5-6 in a transaction that exclusively locks the tuple so that a concurrent cron job doesn't do it.)
postgres  queue  sidekiq  transactions  architecture 
7 days ago by hellsten
Job within a Job: Scheduling Inception with Laravel Queues
Jobs are a super flexible approach to writing domain-specific parts of your application. Pushing the boundaries of Jobs, we can do some really powerful things to automate and simplify our work.

#laravel #queue #jobs #php
Jobs are a super flexible approach to writing domain-specific parts of your application. Pushing the boundaries of Jobs, we can do some really powerful things to automate and simplify our work.

#laravel #queue #jobs #php
laravel  php  queue  cron 
21 days ago by michaelfox
OptimalBits/bull: Premium Queue package for handling jobs and messages in NodeJS.
GitHub is where people build software. More than 28 million people use GitHub to discover, fork, and contribute to over 85 million projects.
nodejs  redis  queue  javascript  performance  reliability  scheduling  retry  concurrency  job 
26 days ago by orlin
Background Processing in Elixir with GenStage – Scripbox Technology Blog – Medium
At Scripbox, we recently started working on a new sub-system which had the requirement to be highly concurrent and distributed. We stumbled on Elixir and found it to be the best tool for the job. We…
asynchronous  elixir  genstage  queue 
26 days ago by nuxlli
Welcome to Django Q — Django Q 0.9.2 documentation
Django Q is a native Django task queue, scheduler and worker application using Python multiprocessing.
Features

Multiprocessing worker pools
Asynchronous tasks
Scheduled and repeated tasks
Encrypted and compressed packages
Failure and success database or cache
Result hooks, groups and chains
Django Admin integration
PaaS compatible with multiple instances
Multi cluster monitor
Redis, Disque, IronMQ, SQS, MongoDB or ORM
Rollbar and Sentry support
django  task  manager  queue  redis  celery  alternative 
26 days ago by christianmlong

Copy this bookmark:





to read