PayPal: Done; Discourse: Done; What’s Next?

I hope this post finds you well. It’s so lovely and sunny here in the UK, it’s (almost) a shame to be inside coding. I guess that’s what laptops are for, right? So you can still code whilst sat in the garden.

The last five weeks have been incredibly busy for me. Aside from starting a new role, I’ve managed to finally finish off a couple of big tasks that have been on my TODO list for waaaay too long. These are:

  • adding PayPal to the site
  • migrating away from Disqus commenting

I want to quickly cover both.

Adding PayPal

If you’ve been getting my mailings / reading the blog for any length of time, you’ll likely be sick of hearing about PayPal. When switching from the old Symfony-only implementation of CodeReviewVideos (version 1), I knew I’d want to offer more than just Stripe as a payment option. Therefore, I dutifully planned ahead and made the process of accepting payments as modular as possible, and made Stripe payment processing just one possible implementation.

This all worked absolutely fine. I was plenty comfortable with Stripe already, and had my original implementation to use as a reference.

What I did wrong, in hindsight, was base my implementation too heavily on Stripe.

To be clear, Stripe get a lot of things right. If you have to accept payments, working with Stripe’s API is a joy. It’s hard not of be influenced by how their system works.

As such, some of the ways I implemented things like the card details endpoint, the invoicing, and even little things like what data was being captured if using a debug level of logging were too heavily tied to Stripe.

The new subscribe form with PayPal option chosen

These things combined to make adding Braintree integration (aka PayPal’s API) take a lot longer than planned (~6 months, to my estimate of about 2-4 weeks). There were some other complications, such as getting my account approved was perhaps something I should have done upfront, but instead, I left this until I was about 8 weeks into development. In hindsight, if they had declined my application, I’d have wasted a lot of time. Not to mention, when I finally thought I might get rejected (it took a while, I got fearful) I stopped development entirely – for about 2 weeks.

The biggest mistake I made though was in the DB schema. Even though I knew upfront that I’d ultimately want to allow people to subscribe with PayPal, or Stripe, I made the relationship between a User account, and a Payment Information a one-to-one.

This was deployed to prod.

All worked fine when all I had was Stripe.

The problem dawned on me that if a User was paying with Stripe, then canceled their subscription, then rejoined with PayPal, then canceled again, and rejoined with Stripe, there was no way to get their previous payment info back. It sounds like an edge case, but if I’ve learned anything from CodeReviewVideos, it’s that all manner of unexpected circumstances can, and do arise. And more frequently than I’d ever have thought.

There was another issue. If a User was paying with Stripe, and then switched to PayPal, with a one-to-one setup they would lose their Stripe invoices. Again, major headaches.

So even when I’d finished the development, I still had a major migration ahead of me. And that consumed about 4 weeks in terms of planning, writing migration scripts, testing, setting up a whole new environment to test the thing end to end… phew, the work just kept on, and on.

Anyway, to cut a long story short, it’s done. The migration went well, and PayPal is now in prod. I think I celebrated by immediately cracking on with the Disqus migration. Ha.

Migrating To Discourse

One aspect of the CodeReviewVideos web site that I’ve never been happy with has been the use of Disqus.

There was a nasty user experience whereby you’d have to sign up once to use the site, and again – and entirely separately – to leave a comment. It sucked. But as far as pragmatic solutions go, it was good enough to get going.

I also read that Disqus would be enabling adverts on my comments section – though to the very best of my knowledge, that never happened. There was talk of a monthly fee. I don’t know. I don’t begrudge them charging for their service, but that wasn’t for me.

Adding Disqus wasn’t super easy, but at the same time, it wasn’t quite as hardcore as the PayPal change.

Surgical

The complications came by way of Single Sign On, hosting Discourse (via Docker), and replacing the existing comments.

Single Sign On wasn’t as bad as I’d expected. I thought that would be the hardest part. I found a Laravel package which I butchered crafted with surgical precision into something that worked nicely with Symfony.

Hosting Discourse wasn’t too bad. I use Rancher for my Docker container management, but Discourse’s Docker implementation just wasn’t playing ball. In the end I got a new VPS from Hetzner and hosted it there instead. There were some tweaks needed, but overall it wasn’t so bad.

Replacing the existing comments was the real tricky part. Disqus provide a one-way export – something I think is a bit weird. By which I mean you can export your data from Disqus, but they won’t let you revert to a previous ‘backup’. Anyway, I didn’t need that, I just needed the export, so that was fine and dandy.

Once I had the export I needed to get that data in to Docker, and then tweak the provided Disqus import script to run against my export. That all generally worked, but it only seems to have missed some of the comments off. I’m not sure why, but also I felt the end result was “good enough”.

The import worked by looking for any existing user, and then mapping a Disqus email address to the user’s Discourse email address. If the Disqus commentor never had a site membership, then now their comment will be assigned to some anonymous username like Disqus_312jklsdf2kl  or whatever. Not perfect, but again, good enough.

Now what happens is when I create a new video, the comments section automagically creates a new forum post under the username chris. As such, if you look at the forum today (and you should, because it’s ace), you’ll see I’ve been posting new topics like a mad man. This will slow down over the next few days.

As I write this I still have email functionality globally disabled on the forum. This will change, possibly over the weekend, once I’m suitably confident everything has settled down. You may recall receiving an email from the staging forum a few weeks back – yep, I made a boob there. Sorry about that. Once bitten, twice shy.

What’s Upcoming

I mentioned at the start that aside from the PayPal and Discourse changes, I have also recently started a new role.

Sometimes I get emails asking why I don’t have any new videos in a while, or why I haven’t updated X, Y, or Z to the latest and greatest. Believe me, I’d love to spend all day making new videos. Unfortunately CodeReviewVideos is not my full time job.

As some of you may know – I’m fairly open about it – I am a contractor by day.

One of the really nice things about being a contractor is getting to experience lots of different projects, in various scales of complexity, and reliability 🙂

I get a lot of interesting ideas for videos from my day-to-day work experiences. And this means that the content on CodeReviewVideos is very much about actionable, real world stuff you can use right now, today in your projects. It’s also battle tested / used in real world production websites.

There’s a bunch of videos I wrote up but haven’t had time to record as of yet. The reason I mention the whole day job thing is that it means I have only a fixed amount of time per week to devote to the site, and I have to prioritise tasks accordingly. Above all else, I prefer making videos. This is why I stared, and continue to run CodeReviewVideos. I love sharing, and from the feedback I get from so many of you (thank you!) you find it useful, too.

But of course, over the last few weeks I’ve had these other big site changes (dare I say, improvements) to make. And that has meant I haven’t been recording. Thankfully all that can change, and I can get back to making recording new stuff.

Ok cool – so what should be being recorded, all being well, is the continuation of the Beginners Guide to Back End (JSON API) + Front End Development [2018] series. Next up is the API Platform section, which I both really enjoyed building, and writing up, and am equally looking forwards to recording and sharing. It’s a good one.

Let’s hope this sunshine continues through the weekend, and for those of you in the UK, enjoy your bank holiday / extended weekend.

One final thing before I go: please do come and say hi on the forum.

Until next time, happy coding!

Chris

Site Update, and GDPR

If you’ve been to the site today you may be wondering just what the heck has changed. And visually, not a lot 🙂 But behind the scenes…

I’ve been working on the PayPal update for a good long while now. I estimated two weeks, and it turned into 6 months, give or take. Fun times.

A big part of why the PayPal update was so long in the making was that I needed to make a major structural change the DB. Originally I had planned for multiple payment providers when switching from the Twig approach, to the new API / hybrid approach.

All that’s cool, and works, but I made a boob in setting the mapping between a User account and a “Payment Information”. I made it one to one, whereas it should have been one to many. Oopsie daisy.

There’s probably a bunch of ways I could have addressed this problem. The way I chose was to have two copies of the database active, change the schema in the prod db, then query the old DB for the relevant info, and write that back to the new schema.

With a change like that I knew there would be downtime. I had a big checklist created on what to do, and how to do it, and largely it all went well.

Somewhere along the way, I decided I also wanted to add a forum. You may have received an email from the forum a few weeks ago. If you did, the link wouldn’t have worked – my mistake, so apologies about that. As it stands, this work is a precursor to the forum work, more on that in another email.

Anyway, I decided to split the PayPal work into two chunks. The first being the DB schema change (now complete), and the second, smaller piece being to enable PayPal. That will happen sometime this weekend. Right now I’m done changing stuff 🙂

The biggest change today has been that now anyone can sign up without needing a credit card. Over the next few weeks, some videos will require you to log in / have a free account to watch.

The existing paid content will remain only available to paying subscribers.

The primary reason for the new free account structure is for the forum.

To be completely clear – for paying subscribers, nothing is changing.


Heads Up: If you experience any strangeness / unexpected errors, please first try logging out and back in again. This update moved the API from Symfony 3 to Symfony 4, and involved a complete new Docker stack. If problems persist, get in touch via the support form, or email.

GDPR

There will be some forthcoming revisions to the terms and conditions, and privacy policy regarding GDPR.

For what it’s worth, I don’t have much tracking. On the site I have Google Analytics (which I use), and Facebook Pixel (which I don’t, but everyone tells me I should).

I also track anonymous video interaction. In other words I know what videos are being watched, and for how long, but I don’t know who by. Personally I don’t like being tracked, and whilst it’s probably detrimental to the success of the site, I have the luxury of building CRV how I want, not how some slimey marketing guy dictates.

The gist of the change is simply this:

You always have the right to request your data. And you always have the right to request I delete any data I have about you.

For existing subscribers who add their name / company details for printing invoices, I wipe these details as and when your paid subscription ends.

Moving forwards I will be deleting inactive user accounts older than 90 days. You will receive an email asking you to log in to confirm you still want your account. If you don’t log in, the account is deleted. With forum integration, this will lead to anonymising your post, but the post will remain.

If you have any questions on this then please reply, email me directly, or use the support form.

Site News

I have been adding videos as regularly as time permits. There have been over 30 new videos uploaded since I last emailed (~6 weeks ago).

Video updates will continue as ever, primarily focusing on the Beginners Guide to Back End (JSON API) + Front End Development [2018] course. I have a bunch of other ad-hoc videos to share, and ~30 new videos written up waited to be recorded.

If you would like to make videos and share them on CodeReviewVideos.com then please get in touch. Beginners are very much welcome, and there are a bunch of benefits to doing this. If at all interested, hit reply and get in touch.

Have A Great Weekend

Ok, that’s about it from me. I’m off out to enjoy the sun.

As ever, have a great weekend, and happy coding.

Chris

[PayPal] 22 Reasons You Can’t Rush The Push

This week saw two more videos added to the Symfony 4 Beginners Tutorial series. There’s only one more video left in this series, which I am going to record over the weekend.

I also will be recording a further ~10 videos on deploying Symfony 4 using a variety of methods. How far I’ll get with these is uncertain at the moment.

As a little bit of insight into my video tutorial creation process:

Each video takes me about three times longer to record than the final length of video would suggest. By which I mean if a video is 5 minutes long, it takes me ~15 minutes to record. The editing takes me anywhere from 2 to 10 minutes per minute of video.

That means for 15 minutes of raw footage I would spend about an hour on the edit. I’d love to make more videos but hopefully that explains why they take a while to release.

My favourite video from this week’s set is this one on AbstractController. I’d love to hear your feedback on it.

Support Changes

I’d like to welcome Joel to the CodeReviewVideos team.

Joel will be handling support issues. As a heads up, Joel is not a Symfony / software developer.

Can I please ask that any technical questions related to video content be left as a comment on the relevant video.

I am working to replace the comments section with an alternative to Disqus. I like the service Disqus offer, but appreciate that having to sign in to two services is not good. A better alternative is coming, but for now, please use the comments system.

PayPal Integration

I mentioned last week that I’m adding PayPal (via Braintree) as a payment method.

I get asked frequently how this is coming along. The answer is: as well as I can expect 🙂

I was a little over optimistic with my due date estimate.

Payment integration is a ton of work. There’s a bunch of obvious tasks:

  • Does the sign up form work with PayPal?
  • Is the payment processed?
  • Is the subscription activated?

And so on.

Then there’s the big list of “oh yeah, I forgot about that” stuff. Things like:

  • Invoices
  • Updating card details for an active subscription
  • “Translating” the errors returned for failed payments into stuff humans understand
  • Not allowing an upgrade from payment Plan A to Plan A
  • Showing the correct proration amount on a payment upgrade

And the like. Some of these issues are small, others are bigger.

During my research into this I found out about services like Chargebee – a one stop shop for this kind of thing. Or in Laravel-land, there is Spark. And in Rails, bullet train.

I was curious as to whether others in Symfony-land would find this kind of thing useful?

The response was sadly underwhelming. I mean I know I have a small number of followers on Twitter, but surely this could be useful to more than just me? Especially given that the other services I mentioned above are $$$.

Wrapping Up

Thanks to everyone who replied to last weeks email. There were a small number interested in seeing videos on PayPal / Stripe integration with Symfony. I’ve added the idea to my list and prioritised according to that feedback.

I’m looking forwards to having finished this PayPal work as I’m itching to get back to writing and recording new content. I have a bunch of topics to cover.

Until next time, have a great weekend and happy coding.

Chris

Symfony 4: Removing the Mystery

The Beginners Symfony 4 tutorial is in progress. I have all but the final video recorded now. I ended up re-recording a number of sections in both of the recent video uploads:

Why?

One of the areas I found most confusing when first starting with Symfony was in the widespread use of Interfaces.

You may have encountered the following problem:

Let’s say you’re working under a tight deadline. You’re writing some Symfony Controller code and working with a form. It’s not going quite as smoothly as you’d like. You reckon something is going awry with the form submission.

Being the inquisitive developer, you remember the oft touted advice:

Read the source, Luke

The thing is, when you ctrl+click on $form->submit($request); you’re taken to… an interface.

This is good stuff.

Your life will be much easier if you code to an interface, rather than tie your methods to specific implementations.

However, with that deadline looming over your shoulder, such things are nice to know, but right now, just show me the code!

Finding An Implementation

When I first recorded this video I initially just said what the outcome of a call to the submit  method would be.

Watching back, I couldn’t help but think about that stuck, and stressed developer. Sat in a noisy office, headphones in, listening to music when you’d rather have peace and quiet.

Everyone around you seems to be goofing off whilst you’re struggling to think through this really important problem.

The last thing you need is to be met with this weird interface  thing. If only you could find the implementation then life would be a lot less stressful.

How can you find out what is really happening when you call $form->submit($request);?

And what happens when you find the implementation and even then the code is tricky to follow?

I know these feels.

That’s why when recording this video I worked hard to make sure you come out at the end with a good understanding of the code that makes this happen.

This is a beginners series for Symfony 4. This is the stuff that will make working with, and understanding Symfony that much easier.

I hope you enjoy it.


Site Stuff

There’s a ton of work going on behind the scenes at the moment.

Can I pay by PayPal?

This is one of the most frequently asked questions that I get.

Currently: no.

That sucks. I appreciate that.

The reasoning for this is that Stripe is super shiny and as a developer, they were high on my wish list of cool things to implement.

Also, from a code perspective Stripe is actually a joy to work with. They really are awesome.

But still, I get asked a lot for PayPal.

I’m adding PayPal.

It’s quite a big job, but I’m about 65% of the way through the implementation.

Here’s a sneak peak:

That’s the Stripe form using Stripe’s React elements.

PayPal functionality is provided through Braintree.

There’s a nice transition between the two options too, which came for free via Bootstrap 4 and I really like it.

I’ve been working on completely extracting the membership code.

Would you have any interest in seeing video tutorials on how to make your own Symfony bundles?

Leave a comment and let me know.

There’s some other cool features that this work enables, which I’ll share with you in a future update.

Until next time, have a great weekend and happy coding.

Chris