this post was submitted on 26 Sep 2024
62 points (98.4% liked)
Python
6368 readers
16 users here now
Welcome to the Python community on the programming.dev Lemmy instance!
π Events
Past
November 2023
- PyCon Ireland 2023, 11-12th
- PyData Tel Aviv 2023 14th
October 2023
- PyConES Canarias 2023, 6-8th
- DjangoCon US 2023, 16-20th (!django π¬)
July 2023
- PyDelhi Meetup, 2nd
- PyCon Israel, 4-5th
- DFW Pythoneers, 6th
- Django Girls Abraka, 6-7th
- SciPy 2023 10-16th, Austin
- IndyPy, 11th
- Leipzig Python User Group, 11th
- Austin Python, 12th
- EuroPython 2023, 17-23rd
- Austin Python: Evening of Coding, 18th
- PyHEP.dev 2023 - "Python in HEP" Developer's Workshop, 25th
August 2023
- PyLadies Dublin, 15th
- EuroSciPy 2023, 14-18th
September 2023
- PyData Amsterdam, 14-16th
- PyCon UK, 22nd - 25th
π Python project:
- Python
- Documentation
- News & Blog
- Python Planet blog aggregator
π Python Community:
- #python IRC for general questions
- #python-dev IRC for CPython developers
- PySlackers Slack channel
- Python Discord server
- Python Weekly newsletters
- Mailing lists
- Forum
β¨ Python Ecosystem:
π Fediverse
Communities
- #python on Mastodon
- c/django on programming.dev
- c/pythorhead on lemmy.dbzer0.com
Projects
- PythΓΆrhead: a Python library for interacting with Lemmy
- Plemmy: a Python package for accessing the Lemmy API
- pylemmy pylemmy enables simple access to Lemmy's API with Python
- mastodon.py, a Python wrapper for the Mastodon API
Feeds
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Hey OP, it looks like you're the author of the post? If so I'm curious how you handle cloud services like AWS or Azure when taking this approach? One of the major issues I've run into when working with teams is how to test or evaluate against cloud services without creating an entire infrastructure in the cloud for testing.
It's a tough one, but there are a few options.
For AWS, my favourite one is LocalStack, a Docker image that you can stand up like any other service and then tell it to emulate common AWS services: S3, Lamda, etc. They claim to support 80 different services which is... nuts. They've got a strange licensing model though, which last time I used it meant that they support some of the more common services for free, but if you want more you gotta pay... and they aren't cheap. I don't know if anything like this exists for Azure.
The next-best choice is to use a stand-in. Many cloud services are just managed+branded Free software projects. RDS is either PostgreSQL or MySQL, ElastiCache is just Redis, etc. For these, you can just stand up a copy of the actual service and since the APIs are identical, you should be fine. Where it gets tricky is when the cloud provider has messed with the API or added functionality that doesn't exist elsewhere. SQS for example is kind of like RabbitMQ but not.
In those cases, it's a question of how your application interacts with this service. If it's by way of an external package (say Celery to SQS for example), then using RabbitMQ locally and SQS in production is probably fine because it's Celery that's managing the distinction and not you. They've done the work of testing compatibility, so theoretically you don't have to.
If however your application is the kind of thing that interacts with this service on a low level, opening a direct connection and speaking its protocol yourself, that's probably not a good idea.
That leaves the third option, which isn't great, but I've done it and it's not so bad: use the cloud service in development. Normally this is done by having separate services spun up per user or even with a role account. When your app writes to an S3 bucket locally, it's actually writing to a real bucket called
companyname-username-projectbucket
. With tools like Terraform, the fiddly process of setting all this up can be drastically simplified, so it's not so bad -- just make sure that the developers are aware of the fact that their actions can incur costs is all.If none of the above are suitable, then it's probably time to stub out the service and then rely more heavily on a QA or staging environment that's better reflective of production.
I appreciate the response!
I've definitely used tools like LocalStack before and when it works it's great, but sadly doesn't usually provide a 1-to-1 replacement.
Seeing your different approaches is helpful and I will have to see what elements I can pull into my current projects!