airflow-balancer 0.7.3
Project description Utilities for tracking hosts and ports and load balancing DAGs Overview airflow-balancer is a utility library for Apache Airflow to track host and port usage via yaml files. It is tightly integrated with airflow-laminar/airflow-config. With airflow-balancer, you can register host and port usage in configuration: _target_: airflow_balancer.BalancerConfiguration default_username: timkpaine hosts: - name: host1 size: 16 os: ubuntu queues: [primary] - name: host2 os: ubuntu size: 16 queues: [workers] - name: host3 os: macos size: 8 queues: [workers] ports: - host: host1 port: 8080 - host_name: host2 port: 8793 Either via airflow-config or directly, you can then select amongst available hosts for use in your DAGs. from airflow_balaner import BalancerConfiguration, load balancer_config: BalancerConfiguration = load("balancer.yaml") host = balancer_config.select_host(queue="workers") port = balancer_config.free_port(host=host) ... operator = SSHOperator(ssh_hook=host.hook(), ...) Visualization Configuration, and Host and Port listing is built into the extension, available either from the topbar in Airflow or as a standalone viewer (via the airflow-balancer-viewer CLI). Installation You can install from pip: pip install airflow-balancer Or via conda: conda install airflow-balancer -c conda-forge License This software is licensed under the Apache 2.0 license. See the LICENSE file for details. Download files Download the file for your platform. If you're not sure which to choose, learn more about installing packages. Source Distribution Built Distribution File details Details for the file airflow_balancer-0.7.3.tar.gz. File metadata Download URL: airflow_balancer-0.7.3.tar.gz Upload date: Jul 2, 2025 Size: 275.3 kB Tags: Source Uploaded using Trusted Publishing? No Uploaded via: twine/6.1.0 CPython/3.12.3 File hashes Hashes for airflow_balancer-0.7.3.tar.gz Algorithm Hash digest SHA256 9e23e4b598247ae72394c45b3c4a0ecef5dfce8c10d5be5173659ff5322d8341 MD5 a70d806f720bdb53a413a2bdc1049f87 BLAKE2b-256 48cc16a2255fcc4d512d0f550105780a7d87f49c5b093b8ea6bc684186012420 See more details on using hashes here. File details Details for the file airflow_balancer-0.7.3-py3-none-any.whl. File metadata File hashes Hashes for airflow_balancer-0.7.3-py3-none-any.whl Algorithm Hash digest SHA256 6a32344b81155cc28840610804622447f480f0b394d48bb7bb8a3015e8d88cec MD5 68db971d435df374214a09577aace478 BLAKE2b-256 735edf956f4d0a0609d6439bb09f13525649d173f3919a5e5fa1374e9a939160 See more details on using hashes here.