Production ready Orchestration Framework to manage product lifecycle and workflows. Easy to use, Built on top of FastAPI
The orchestrator core is a project developed by SURF to facilitate the orchestration of services. Together with ESnet this project has been opensourced in the commons conservancy to help facilitate collaboration. We invite all who are interested to take a look and to contribute!
When do you orchestrate and when do you automate? The answer is you probably need both. Automation helps you execute repetitive tasks reliably and easily, Orchestration adds a layer and allows you to add more intelligence to the tasks you need to automate and to have a complete audit log of changes.
Orchestrate* - Transitive Verb
1: Arrange or score (music) for orchestral performance.
‘the song cycle was stunningly arranged and orchestrated’
2: Arrange or direct the elements of (a situation) to produce a desired effect, especially surreptitiously.
‘the developers were able to orchestrate a favorable media campaign’
This orchestrator-core provides a framework through which you can manage service orchestration for your end users. It enables you to define products to which users can subscribe, and helps you intelligently manage the lifecycle of Creation, Modification, Termination and Validation of a customer subscription.
This project can be installed as follows:
Install the core.
pip install orchestrator-core
Create a postgres database:
createuser -sP nwa createdb orchestrator-core -O nwa
Step 3 (optional):
When using multiple workers, you will need a redis server for live updates with websockets.
By default it will use memory which works with only one worker.
For the redis connection you need to set the env variable with the connection url.
Websockets can also be turned off with:
More broadcaster info here
If you want to use pickle for CACHE serialization you will need to set the
from orchestrator import OrchestratorCore from orchestrator.cli.main import app as core_cli from orchestrator.settings import AppSettings app = OrchestratorCore(base_settings=AppSettings()) if __name__ == "__main__": core_cli()
Step 5 (Optional):
OrchestratorCore comes with a graphql interface that can to be registered after you create your OrchestratorApp.
If you add it after registering your
SUBSCRIPTION_MODEL_REGISTRY it will automatically create graphql types for them.
More info can be found in
from orchestrator import OrchestratorCore from orchestrator.settings import AppSettings app = OrchestratorCore(base_settings=AppSettings()) # register SUBSCRIPTION_MODEL_REGISTRY app.register_graphql()
Initialize the migration environment.
PYTHONPATH=. python main.py db init PYTHONPATH=. python main.py db upgrade heads
Authentication and authorization are default enabled, to disable set
uvicorn --reload --host 127.0.0.1 --port 8080 main:app
Visit http://127.0.0.1:8080/api/redoc to view the api documentation.
Setting up a development environment
To add features to the repository follow the following procedure to setup a working development environment.
Installation (Development standalone)
Install the project and its dependencies to develop on the code.
Step 1 - install flit:
python3 -m venv venv source venv/bin/activate pip install flit
Step 2 - install the development code:
Make sure to use the flit binary that is installed in your environment. You can check the correct path by running
To be sure that the packages will be installed against the correct venv you can also prepend the python interpreter that you want to use:
flit install --deps develop --symlink --python venv/bin/python
Run the unit-test suite to verify a correct setup.
Step 1 - Create a database
createuser -sP nwa createdb orchestrator-core-test -O nwa
Step 2 - Run tests
pytest -n auto test/unit_tests
If you do not encounter any failures in the test, you should be able to develop features in the orchestrator-core.
Installation (Development symlinked into orchestrator SURF)
If you are working on a project that already uses the
orchestrator-core and you want to test your new core features
against it, you can use some
flit magic to symlink the dev version of the core to your project. It will
automatically replace the pypi dep with a symlink to the development version
of the core and update/downgrade all required packages in your own orchestrator project.
Step 1 - install flit:
python - m venv venv source venv/bin/activate pip install flit
Step 2 - symlink the core to your own project
flit install --deps develop --symlink --python /path/to/a/orchestrator-project/venv/bin/python
So if you have the core and your own orchestrator project repo in the same folder and the main project folder is
orchestrator and you want to use relative links, this will be last step:
flit install --deps develop --symlink --python ../orchestrator/venv/bin/python
Increasing the version number for a (pre) release.
When your PR is accepted you will get a version number.
You can do the necessary change with a clean, e.g. every change committed, branch:
bumpversion patch --new-version 0.4.1-rc3