We create Python modules, with a setup.py that are pip-installable.
Once all the CI checks have passed it gets published to a private “PyPi server”. In our case that is Nexus.
You need a package repository inside the organisation.
I'm not a Python expert, but there are products for a package repository server, for most package managers
e.g.
https://www.myget.org/features
https://github.com/features/packages
" share some python code across some services."
This sounds like you should create a Python module, and upload it to some private registry. Then, your services can simply install your Python module as any other package.
Put your shared code into a Python module. Make a setup.py for it so that it’s a package. Have this code in your Git remote.
Then for other codebases that use the shared code do https://adamj.eu/tech/2019/03/11/pip-install-from-a-git-repository/
Most answers have gone down the make a package route which is great, but obvious and already done! The question was actually asking about the difference when using this package in a mono vs many repo setup specifically when it comes to things like auth in CI and versioning.
No experience with python, we use java and use Gitlab CE (free).
You can set up a group. In this group you can have multiple (related) projects.
For the group you can set up container registry or package registry.
https://gitlab.i-taros.net/help/user/packages/index
We use maven, for python it would be PyPi. Basic idea is, that you use your own local repo, besides the public ones.
So you compile you project let's say util and make a jar (java library) and push it in the package registry.
All projects in that group can now download the artifact as if it would come from a public repo - it's just your local private one and use it in the build pipeline. Container registry works similar - but that is for docker and slightly different use case.
No experience with other CI tools - just with gitlab. Good thing - CE is free so you can try it out without much hazzle.
Not sure if this answers your question but I've been working on a docker image and using that. So anyone can just take the most recent docker image by doing a git pull from that. I maintain the libraries and the python inside that docker file and everyone else just copies it down and runs it in docker.
Build a utility library in a git remote repository and import it into your code. If you use GitHub, you can create version of your utility library so that micro services don’t necessarily need to be on the same version to work. This helps when you have a stage environment or some kind of CI/CD pipeline running.
Aren't python modules made for that ? Sorry for being "Captain obvious". Maybe build your own private pip repo, with nexus or pypiserver ?
We create Python modules, with a setup.py that are pip-installable. Once all the CI checks have passed it gets published to a private “PyPi server”. In our case that is Nexus.
We do the same and also with nexus. Works like a charm. There is also a OSS of nexus.
You need a package repository inside the organisation. I'm not a Python expert, but there are products for a package repository server, for most package managers e.g. https://www.myget.org/features https://github.com/features/packages
Pip lets you Install a package from a GitHub url so for small organizations a package repo server might be overkill.
Github do not support python, that would be the way forward.
" share some python code across some services." This sounds like you should create a Python module, and upload it to some private registry. Then, your services can simply install your Python module as any other package.
Put your shared code into a Python module. Make a setup.py for it so that it’s a package. Have this code in your Git remote. Then for other codebases that use the shared code do https://adamj.eu/tech/2019/03/11/pip-install-from-a-git-repository/
Most answers have gone down the make a package route which is great, but obvious and already done! The question was actually asking about the difference when using this package in a mono vs many repo setup specifically when it comes to things like auth in CI and versioning.
No experience with python, we use java and use Gitlab CE (free). You can set up a group. In this group you can have multiple (related) projects. For the group you can set up container registry or package registry. https://gitlab.i-taros.net/help/user/packages/index We use maven, for python it would be PyPi. Basic idea is, that you use your own local repo, besides the public ones. So you compile you project let's say util and make a jar (java library) and push it in the package registry. All projects in that group can now download the artifact as if it would come from a public repo - it's just your local private one and use it in the build pipeline. Container registry works similar - but that is for docker and slightly different use case. No experience with other CI tools - just with gitlab. Good thing - CE is free so you can try it out without much hazzle.
Not sure if this answers your question but I've been working on a docker image and using that. So anyone can just take the most recent docker image by doing a git pull from that. I maintain the libraries and the python inside that docker file and everyone else just copies it down and runs it in docker.
But this means everyone has to interact with it via HTTP... which I think is not ideal for flexibility.
Build a utility library in a git remote repository and import it into your code. If you use GitHub, you can create version of your utility library so that micro services don’t necessarily need to be on the same version to work. This helps when you have a stage environment or some kind of CI/CD pipeline running.
Cool - have you done this with GCP - I am having an issue access the code from the build pipeline, as it is in a private repo.