May 1 Transactions in Queue

Lindsey Long's Avatar

Lindsey Long

02 May, 2018 02:25 PM

Hello support team,
Just checking in to see if there are some issues with card processing yesterday. I see several still in the queue. Please let me know.

Thank you,
Lindsey

  1. Support Staff 1 Posted by Meghan Turner on 02 May, 2018 03:08 PM

    Meghan Turner's Avatar

    Hi Lindsey,

    We're investigating, get back to you with more information shortly!

    Meghan

  2. 2 Posted by Lindsey Long on 02 May, 2018 05:31 PM

    Lindsey Long's Avatar

    Hi Meghan,

    Could you provide any update you may have? I'm noticing that nothing is processing today as well. I want to reach out to Authorize if this is something that is happening on their side. Let me know.

    Thanks,
    Lindsey

  3. Support Staff 3 Posted by Meghan Turner on 02 May, 2018 05:39 PM

    Meghan Turner's Avatar

    Hi Lindsey,

    It looks like the problem is on our end this time. We've identified an issue that occurred during recent database maintenance that was causing invoices on the 1st of the month to be queued rather than transacted. Our team is working on repairing that issue as soon as possible. Once a fix is in place, we'll be able to run those transactions for you. I'll keep you updated on our progress.

    So sorry for the delay in processing! I know that you need these transactions to come through as quickly as possible, so we'll do everything we can to get them running again soon.

    Meghan

  4. 4 Posted by Lindsey Long on 02 May, 2018 05:42 PM

    Lindsey Long's Avatar

    Ok thank you for the update!

  5. 5 Posted by Lindsey Long on 02 May, 2018 09:53 PM

    Lindsey Long's Avatar

    Hey Meghan,
    Just checking on an update since we're at the end of the day. Are we the only customer having this issue or is this system wide?

    Lindsey

  6. 6 Posted by Michael Sanders on 02 May, 2018 10:17 PM

    Michael Sanders's Avatar

    Lindsey, you're not the only one. We're experiencing an issue too.

  7. 7 Posted by Lindsey Long on 02 May, 2018 10:19 PM

    Lindsey Long's Avatar

    Thank you Michael! That's a little discouraging.

  8. Support Staff 8 Posted by Meghan Turner on 03 May, 2018 04:16 AM

    Meghan Turner's Avatar

    Hey Lindsey and Michael,

    Quick update for you both. In order to perform essential maintenance on our database today to repair the issue I mentioned earlier, we had to disable the recurring engine. When the recurring engine is disabled, invoices do not automatically transact. Any invoices that were due while the recurring engine was disabled will remain in a queued state. The recurring engine is fault tolerant, so when it's re-enabled, it will transact any queued invoices that didn’t process while the engine was suspended.

    Our team of engineers and database administrators has been working non-stop today to get this resolved because we know it's incredibly important to our merchants that transactions are running as expected. We're in the final stages of implementing a fix and anticipate being able to re-enable the recurring engine soon.

    In the meantime, it is possible to manually run queued transactions even when the recurring engine is disabled. If there are queued invoices you'd like to go ahead and transact now, click on the queued invoice and hit 'run invoice' in the bottom left corner of the page.

    Also, we've recently launched a status page for Cheddar and we'll be posting updates on this issue there. I'll continue to update you via this support discussion, but if you'd like to monitor the status page as well check out https://status.getcheddar.com/.

    Meghan

  9. 9 Posted by Lindsey Long on 03 May, 2018 02:16 PM

    Lindsey Long's Avatar

    Meghan,
    Is there a way to pull a report that will show us everything that is currently queued so we can make sure we get everything?

    Lindsey

  10. Support Staff 10 Posted by Meghan Turner on 03 May, 2018 02:52 PM

    Meghan Turner's Avatar

    Hi Lindsey,

    I've been using the customer's list in the Cheddar dashboard to pull information about who has queued invoices. Specifically, I'm getting the Active and on Paid Plan list, then sorting by invoice due date by clicking on the 'next invoice' column heading. That sorts the customer list by either furthest invoice in the future or most recent invoice.

    Anyone with an invoice date in the past (i.e. 1-2 days ago) most likely has a queued invoice on their account.

    Meghan

  11. 11 Posted by Marcela Poffald on 03 May, 2018 07:11 PM

    Marcela Poffald's Avatar

    Hi Lindsey and Michael,

    We've just done a one-time run of the recurring engine on both of your products, which should take care of all the transactions that were in a queued state as of now. The recurring engine still isn't running automatically yet, but this should have run all the invoices that were queued from the last few days.

    We'll keep you posted on the status of the recurring engine as we continue to work on the issue.

    Marcela

  12. 12 Posted by Marcela Poffald on 03 May, 2018 09:17 PM

    Marcela Poffald's Avatar

    Hi Lindsey and Michael,

    I apologize for the delay in processing, the recurring engine is now back online and transactions are running automatically again.

    We'll be doing a postmortem analysis of the incident over the next day or so, and we'll be posting the results on our status page at https://status.getcheddar.com/, if you're interested.

    If you have any other questions or if there's anything else we can help you with, please let us know!

    Marcela

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac