Would you use Symfony for SaaS?

Over the past four weeks I have been working on the revised back end for CodeReviewVideos. The part that’s taken the longest has been working with payment / billing integration.

Over the past few years a steady stream of requests come in asking if it’s possible to pay with PayPal. Up until now I have had no streamlined way to accept PayPal, but with the new back end this should be possible. One thing though, it might not be there on day 1.

I’m not particularly shy about my reasons for not having any other payment option than Stripe. The biggy would be that Stripe is amazing to work with, and as a developer it’s by far and away the nicest payment integration I’ve ever dealt with.

But the primary reason I don’t have PayPal so far is that adding in PayPal – or any other payment provider – to the existing setup for CodeReviewVideos is really, really hard.

When I first came to launch the site, I had no idea how it would perform. Would there be 10 people wanting to join, 100, 1000… or none?

Aside from all that, I had a ton of other stuff to get right for the launch.

Rightly or wrongly, my first priority has always been to create new content. I think about my preferences as a visitor of other sites that provide online programming training videos, and I can forgive a heck of a lot if the content is valuable to me.

All of this is a roundabout way of me saying I pretty much ‘hardcoded’ Stripe as the only payment provider, and that made it super hard to extract Stripe and make the concept of payment more abstract.

I’ve since found out (as per the 4 weeks above) that I probably made the right choice.

If no one had subscribed, that would have been four weeks of totally lost effort – on top of the months I’d already put in.

But this got me to thinking further: why isn’t this a pre-solved problem? Surely I’m not the only one using Symfony in this way?

There’s a bunch of really fiddly bits to get right when working with payments. It’s got to be absolutely perfect, or people – rightfully – get mad, and / or lose a lot of confidence. Imagine if the first encounter you have with a site is the sign up process breaking for whatever reason – you’re not going to feel too confident about the rest of the site after that. At least, that’s how I feel.

What this all boils down to is that the whole process needs to be as rigorously tested as possible. But writing these tests is a total pain.

Normally when testing the “golden rule” is to avoid any real calls to external systems.

With payment though, this is a total catch 22.

If I don’t interact with e.g. Stripe during tests then I need to keep on top of any API changes they make at a microscopic level. If they change the way they send events, I need to update all my fake data, validate everything, and even then I’m really just “resetting” the problem until they next change something.

Now, in truth, Stripe are decent about this and they won’t force an API update on you – at least not right away. But still, it’s not what I wanted to do.

Instead, I went with tests that talk to Stripe. Even this isn’t perfect – some events are not easily faked. An example of this is when a customer’s payment fails. Stripe will retry this payment attempt a few times before sending the webhook to my configured endpoint with the invoice.payment_failed notification.

What this means is I had to fake that one particular event. All the others actually talk to Stripe.

Another downside to this is that when faking an event, the given id field will be very fake – e.g. evt_0000000000000 or similar.

Stripe suggest each event received via a webhook should be considered untrusted. The solution here is to take the received ID and then call Stripe saying hey, is this ID actually legit? Of course, evt_0000000000 is not legit, which means either hackery in my code, or avoid making the validation call.

Needless to say, getting through this process felt like an absolute grind. Getting the tests to a point where they don’t flake out when run via GitLab CI was amongst the trickiest parts.

Whilst in this grind phase I was having a bit of a back-and-forth chat with a fellow dev and the conversation touched on two interesting questions:

  1. How many other developers are using Symfony for SaaS, or SaaS-like services?
  2. Is Stripe / payment integration something that many other devs want or need tutorial content on?

I’d be really grateful if you could hit reply and let me know your thoughts on these two points.

I will share my own opinions on these two points next week.

I think that’s the hardest part done now. There’s a couple of steps that remain before I can make the big switch over, but it’s almost there.

Video Update

This week saw three new videos added to the site:

https://codereviewvideos.com/course/let-s-build-a-wallpaper-website-in-symfony-3/video/fixing-the-fixtures

Early in our project we created Doctrine Fixtures to setup our database with known data. We now need to keep our fixtures working with the latest changes.

https://codereviewvideos.com/course/let-s-build-a-wallpaper-website-in-symfony-3/video/untested-updates

Learn how to accept Updates to existing uploaded Wallpaper image files in this first of three parts covering Update and Delete in EasyAdminBundle.

https://codereviewvideos.com/course/let-s-build-a-wallpaper-website-in-symfony-3/video/untested-updates-part-two-now-we-can-actually-update

There’s a bunch of little gotchas that have caught us out along the way to a working Update process. By the end of this video we should have fixed them all.

We’re nearly at the end of the first phase of the Wallpaper Website tutorial. I’ve had a few video requests come in whilst recording this series so we will move on to addressing them, along with a couple of other series that are constantly requested, before moving on to phase two.

Thanks for reading, have a great weekend, and happy coding.

Chris

A Tweak For ELK On Docker

One annoyance I’ve hit whilst running ELK on Docker is, after rebooting my system, the same error keeps returning:

The core of the fix to this problem is helpfully included in the output:

And on Ubuntu fixing this is a one-liner:

Only, this is not persisted across a reboot.

To fix this permenantly, I needed to do:

sudo vim /etc/sysctl.d/60-elasticsearch.conf

And add in the following line:

I am not claiming credit for this fix. I found it here. I’m just sharing here because I know in future I will need to do this again, and it’s easiest if I know where to start looking 🙂

Another important step if persisting data is to ensure the folder is owned by 991:991 .

Docker or The Hard Way

Last week I asked the following question:

Thinking about a project where you would need to use RabbitMQ, or Redis, or the ELK stack, or any other potentially complex external software, how would you prefer to cover the setup of this stack?

I know this mail out doesn’t go to a huge number of people, but I was hoping for a slightly better response rate than what I received.

As it stands I got 4 replies.

Sad trombone.

As the survey was anonymous, I unfortunately cannot personally contact each respondent to say thank you – so if you did reply then sincerely, thank you 🙂

Still, here are the results:

This is still very useful to me. It’s roughly in line with what I was thinking.

Let’s take a video series on integrating RabbitMQ with Symfony as an example.

There’s broadly three parts (in my mind) to a series like this:

  • Server setup
  • Basic usage
  • Advanced usage

For me, the hardest part is the server setup phase.

There’s a ton of ways to setup a server. To what level do I cover this part? Should it be good enough to get started, or a complete build that’s secure and ready for production?

Docker is fairly straightforward (to a point) – something like me sharing a docker-compose.yml  file, and then you running  docker-compose up. That assumes you’re comfortable with Docker, and have Docker install, of course 🙂

Manual set up is a little more involved. What OS are we installing on? Will it be on a VM, or on a VPS like Digital Ocean or Linode? How to do we reproduce this build when moving into production? Etc.

Anyway, lots of new things for me to ponder. I really do appreciate knowing this stuff, even if it does open up further questions.

Again, thank you 🙂

Video Update

This week saw three new videos added to the site.

We are continuing on with our Wallpaper website, specifically this week wrapping up the test-driven approach to wallpaper file uploads.

I’m genuinely curious as to your opinions on this process so far. I find it somewhat amusing that we covered wallpaper file uploads without tests in just two videos, and the very same process when testing took five videos.

The underlying differences and resulting system design are definitely different. But are they different good, or different bad? Is the system now easier or harder to understand?

These aren’t questions that I have a definitive answer too, by the way.

Next up we are going to repeat this process for both ‘Edit’ and ‘Delete’.

This will bring us almost to a close of the first part of this series.

Until next week, have a great weekend and happy coding.

Chris

Put Your Skills To The Test

Testing is an interesting subject. I’d say it’s a fairly safe bet to say that we all believe we should be writing tests. However, from personal real-world experience, far fewer developers actively write tests than I would have ever thought.

This often gets me to wondering: Why?

Why – if we know that well-tested code is easier to work with, easier to change, and generally makes our lives easier – do we choose not to write tests?

“Time” is probably the easiest thing to blame. I’ve lost track of the number of company stand-ups I’ve attended where a developer will mention testing in some capacity, and an antsy project manager will espouse the following wisdom:

Let’s skip the testing for now, and get this feature done. We can come back and add tests later.

If only we had a remote control to fast-forward time. We could quickly visit the future stand-up where the bugs introduced as a result of this PM’s (complete lack of) years of programming wisdom and skill came back to bite the entire team on the backside.

Time is easy to blame because it’s not about exposing our lack of understanding of something technical.

I’m sure you’ve been there: part of a conversation about some programming concept that, even though you’ve never heard of up until now, for some inexplicable reason, you suddenly strongly believe you should already be a fully-versed expert in. The correct approach here is to nod and make affirmative noises, then excuse yourself, lock yourself in the toilets and frantically StackOverflow all these new terms on your smart phone.

No, not really.

There’s so much to know as part of our job as a software developer. It’s a constant drive to self-improve.

And the truth is: there’s too much stuff to know. You cannot feasibly be expected to know all of it.

But that’s no excuse to avoid learning new stuff that does matter to getting your job done in the most efficient and effective manner.

For me, this means that generally my development life will be easier and less stressful.

How the heck does any of this relate to testing?

Well, to understand that let’s forget all about PHP for a moment. We’re going to choose a completely different language:

F#.

I’m hoping I’ve chosen a language here that you don’t know. If you do already know F# then you get a free re-roll, and you can magically substitute this for a language you don’t already know.

Let’s imagine that we’ve been learning a little bit here and there about F# in our spare time.

As part of a different project, some particular problem has come up that we are confident F# is the correct language in which to solve it.

The thing is, our knowledge of F# is firmly rooted in the “beginner” tier.

But our skill set as a PHP developer tells us that if we’re going to be writing code, we should probably be writing some tests.

At this point we have a few options:

  • Don’t test
  • Accept that figuring out how to test is going to slow down our prototype significantly, and resolve to pay that price
  • Don’t use F#

I guess you could also outsource this to some high priced F# contractor.

The easiest approach here not to use F#.

We could likely fudge our way through whatever challenge we face by making use of the skills we already have. I’m sure you’ve encountered code like this before.

I’m not exactly sure how to find a Higgs Boson, but I reckon the best place to start is with PHP.

But this approach sucks. You’ve likely seen the advice: use the right tool for the job. PHP is often not the right tool for the job, but it gets used anyway. For every day evidence of this, just look to WordPress, where a blogging platform is being used as a help desk, business listing directory, or a shop. Heck, if you look long enough, I bet you’d find a WordPress website out there right now that’s doing all three.

By the way, that’s not a slight on WordPress which I think in itself is a brilliantly useful tool.

This leaves us with two options: Don’t test, or … learn how to test.

Skipping the tests will likely yield you a working system in a much faster time frame.

It’s when you need to continually work on, improve, and generally maintain your code that a lack of tests really bites you. It’s during this phase that all that time you saved up front is meticulously eaten away until suddenly, you’re on the wrong side of the curve and it’s a very long way back.

But I get it.

I mean – we barely know F#. How on Earth can we be expected to know how to write testable F# code?

It’s a really, really tricky spot to be in. I know, I’ve found myself in this very same spot in a variety of languages – from PHP to Python, to more recently Elixir.

Personally I’ve found it really hard to mentally accept that I don’t know enough to achieve what my mind is telling me should be really quite straightforward.

Ok, enough F#.

Back To PHP

I’ve found it really easy to forgive myself about a lack of testing when writing code in any language other than PHP.

It’s hard enough to learn a new language, let alone having the burden of writing some tests to prove my stuff works as expected.

But I’m going to assume you are past the absolute beginner phase when it comes to PHP. You feel comfortable with the syntax, and now you want to continue your learning by seeing what Symfony has to offer.

Even though you know PHP, and maybe you know how to write tests in PHP, applying that knowledge whilst learning Symfony kinda feels like we are back in that F# position again. It’s all new, and that can be incredibly overwhelming.

So here’s what we are going to do:

We are learning us some Symfony. That much should be obvious, given what CodeReviewVideos is predominantly about.

Along the way we are learning how to do common “stuff” with Symfony. Forms, database interaction, services, all that jazz.

Once we’ve seen how all this stuff works, we are ripping it out again and having another crack at it. Only this time, we are going to learn how to do it with tests.

In this week’s videos we are covering how to use PhpSpec to guide ourselves through this process.

If you haven’t already done so, I’d strongly recommend starting with last week’s videos.

There are a bunch of new concepts and techniques to learn here.

It’s true, testing isn’t easy.

If it were, we would probably all do it much more than we actually seem to do.

This week’s videos follow consecutively. To get the most of out of them, you really need to have been following along, at least since video 19 – No Tests – Part 1 – Uploading Files in EasyAdminBundle.

There’s really no excuse on this – all of these videos are completely free. I’d love to hear your feedback – as ever please either reply here, or on the video your questions or feedback directly relate too.

Whilst testing is initially hard to get started with, even having just one or two test files in your project is hugely beneficial to adding more (think: copy paste! although with PhpSpec, there’s a better way than that.)

Following Along

There’s a problem that I’d really appreciate your help with this week:

Thinking about a project where you would need to use RabbitMQ, or Redis, or the ELK stack, or any other potentially complex external software, how would you prefer to cover the setup of this stack?

I would really appreciate it if you could answer this one question survey.

I am running this survey until next Friday at 10AM GMT. After which, I shall share the results in next week’s newsletter.

As ever, thank you very much for reading, I hope you enjoy this week’s new videos, and that you have a great weekend.

Until next week, happy coding!

Chris

Stripe Webhooks List

I recently needed access to a list of Stripe’s available webhooks.

Stripe have a list, but I found my eyes became extremely fatigued when referring to their list. So I copy pasted, and restyled to something I prefer.

That is all 🙂

Account

account.updated
Occurs whenever an account status or property has changed.
account.application.deauthorized
Occurs whenever a user deauthorizes an application. Sent to the related application only.
account.external_account.created
Occurs whenever an external account is created.
account.external_account.deleted
Occurs whenever an external account is deleted.
account.external_account.updated
Occurs whenever an external account is updated.

Application Fee

application_fee.created
Occurs whenever an application fee is created on a charge.
application_fee.refunded
Occurs whenever an application fee is refunded, whether from refunding a charge or from refunding the application fee directly, including partial refunds.
application_fee.refund.updated
Occurs whenever an application fee refund is updated.

Balance

balance.available
Occurs whenever your Stripe balance has been updated (e.g., when a charge is available to be paid out). By default, Stripe automatically transfers funds in your balance to your bank account on a daily basis.

Bitcoin

bitcoin.receiver.created
Occurs whenever a receiver has been created.
bitcoin.receiver.filled
Occurs whenever a receiver is filled (i.e., when it has received enough bitcoin to process a payment of the same amount).
bitcoin.receiver.updated
Occurs whenever a receiver is updated.
bitcoin.receiver.transaction.created
Occurs whenever bitcoin is pushed to a receiver.

Charge

charge.captured
Occurs whenever a previously uncaptured charge is captured.
charge.failed
Occurs whenever a failed charge attempt occurs.
charge.pending
Occurs whenever a pending charge is created.
charge.refunded
Occurs whenever a charge is refunded, including partial refunds.
charge.succeeded
Occurs whenever a new charge is created and is successful.
charge.updated
Occurs whenever a charge description or metadata is updated.
charge.dispute.closed
Occurs when a dispute is closed and the dispute status changes to charge_refunded, lost, warning_closed, or won.
charge.dispute.created
Occurs whenever a customer disputes a charge with their bank.
charge.dispute.funds_reinstated
Occurs when funds are reinstated to your account after a dispute is won.
charge.dispute.funds_withdrawn
Occurs when funds are removed from your account due to a dispute.
charge.dispute.updated
Occurs when the dispute is updated (usually with evidence).
charge.refund.updated
Occurs whenever a refund is updated on selected payment methods.

Coupon

coupon.created
Occurs whenever a coupon is created.
coupon.deleted
Occurs whenever a coupon is deleted.
coupon.updated
Occurs whenever a coupon is updated.

Customer

customer.created
Occurs whenever a new customer is created.
customer.deleted
Occurs whenever a customer is deleted.
customer.updated
Occurs whenever any property of a customer changes.
customer.discount.created
Occurs whenever a coupon is attached to a customer.
customer.discount.deleted
Occurs whenever a coupon is removed from a customer.
customer.discount.updated
Occurs whenever a customer is switched from one coupon to another.
customer.source.created
Occurs whenever a new source is created for a customer.
customer.source.deleted
Occurs whenever a source is removed from a customer.
customer.source.updated
Occurs whenever a source’s details are changed.
customer.subscription.created
Occurs whenever a customer is signed up for a new plan.
customer.subscription.deleted
Occurs whenever a customer’s subscription ends.
customer.subscription.trial_will_end
Occurs three days before the trial period of a subscription is scheduled to end.
customer.subscription.updated
Occurs whenever a subscription changes (e.g., switching from one plan to another or changing the status from trial to active).

Invoice

invoice.created
Occurs whenever a new invoice is created. See Using Webhooks with Subscriptions to learn how webhooks can be used with, and affect, this event.
invoice.payment_failed
Occurs whenever an invoice payment attempt fails, either due to a declined payment or the lack of a stored payment method.
invoice.payment_succeeded
Occurs whenever an invoice payment attempt succeeds.
invoice.upcoming
Occurs X number of days before a subscription is scheduled to create an invoice that is charged automatically, where X is determined by your subscriptions settings.
invoice.updated
Occurs whenever an invoice changes (e.g., the invoice amount).
invoiceitem.created
Occurs whenever an invoice item is created.
invoiceitem.deleted
Occurs whenever an invoice item is deleted.
invoiceitem.updated
Occurs whenever an invoice item is updated.

Order

order.created
Occurs whenever an order is created.
order.payment_failed
Occurs whenever an order payment attempt fails.
order.payment_succeeded
Occurs whenever an order payment attempt succeeds.
order.updated
Occurs whenever an order is updated.
order_return.created
Occurs whenever an order return is created.

Payout

payout.canceled
Occurs whenever a payout is canceled.
payout.created
Occurs whenever a payout is created.
payout.failed
Occurs whenever a payout attempt fails.
payout.paid
Occurs whenever a payout is expected to be available in the destination account. If the payout fails, a payout.failed notification is additionally sent at a later time.
payout.updated
Occurs whenever the metadata of a payout is updated.

Plan

plan.created
Occurs whenever a plan is created.
plan.deleted
Occurs whenever a plan is deleted.
plan.updated
Occurs whenever a plan is updated.

Product

product.created
Occurs whenever a product is created.
product.deleted
Occurs whenever a product is deleted.
product.updated
Occurs whenever a product is updated.

Recipient

recipient.created
Occurs whenever a recipient is created.
recipient.deleted
Occurs whenever a recipient is deleted.
recipient.updated
Occurs whenever a recipient is updated.

Review

review.closed
Occurs whenever a review is closed. The review’s reason field indicates why (e.g., approved, refunded, refunded_as_fraud, disputed.
review.opened
Occurs whenever a review is opened.

SKU

sku.created
Occurs whenever a SKU is created.
sku.deleted
Occurs whenever a SKU is deleted.
sku.updated
Occurs whenever a SKU is updated.

Source

source.canceled
Occurs whenever a source is canceled.
source.chargeable
Occurs whenever a source transitions to chargeable.
source.failed
Occurs whenever a source fails.
source.transaction.created
Occurs whenever a source transaction is created.

Transfer

transfer.created
Occurs whenever a transfer is created.
transfer.reversed
Occurs whenever a transfer is reversed, including partial reversals.
transfer.updated
Occurs whenever the description or metadata of a transfer is updated.

Misc

ping
May be sent by Stripe at any time to see if a provided webhook URL is working.