Intro

This tutorial will show you how to use FastKafkaAPI, step by step.

The goal of FastKafkaAPI is to simplify the use of Apache Kafka in Python inspired by FastAPI look and feel.

In this Intro tutorial we’ll go trough the basic requirements to run the demos presented in future steps.

Installing FastKafkaAPI

First step is to install FastKafkaAPI

$ pip install fastkafka

Preparing a Kafka broker

Next step is to prepare the Kafka environment, our consumers and producers will need some channel of communication.

!!! info "Hey, your first info!"

If you already have an instance of Kafka running that you can connect to for demo purposes, feel free to skip this step. 

To go through the tutorial, we recommend that you use dockerized Kafka brokers, if you have Docker and docker-compose installed the setup should take you no time (if we exclude the container download times).

!!! warning "Listen! This is important."

To be able to setup this configuration you need to have Docker and docker-compose installed

See here for more info on <a href = \"https://docs.docker.com/\" target=\"_blank\">Docker</a> and <a href = \"https://docs.docker.com/compose/install/\" target=\"_blank\">docker compose</a>

To setup the recommended environment, first, create a new folder wher you want to save your demo files (e.g. fastkafka_demo). Inside the new folder create a new YAML file named kafka_demo.yml and copy the following configuration into it:

version: "3"
services:
    zookeeper:
        image: wurstmeister/zookeeper
        hostname: zookeeper
        container_name: zookeeper
        networks:
          - fastkafka-network
        ports:
          - "2181:2181"
          - "22:22"
          - "2888:2888"
          - "3888:3888"
    kafka:
        image: wurstmeister/kafka
        container_name: kafka
        ports:
          - "9093:9093"
        environment:
            HOSTNAME_COMMAND: "docker info | grep ^Name: | cut -d' ' -f 2"
            KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181"
            KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTER:PLAINTEXT,INSIDE:PLAINTEXT
            KAFKA_ADVERTISED_LISTENERS: INTER://:9092,INSIDE://localhost:9093
            KAFKA_LISTENERS: INTER://_{HOSTNAME_COMMAND}:9092,INSIDE://:9093
            KAFKA_INTER_BROKER_LISTENER_NAME: INTER
            KAFKA_CREATE_TOPICS: "hello:1:1"
        volumes:
            - /var/run/docker.sock:/var/run/docker.sock
        depends_on:
            - zookeeper
        healthcheck:
            test: [ "CMD", "kafka-topics.sh", "--list", "--zookeeper", "zookeeper:2181" ]
            interval: 5s
            timeout: 10s
            retries: 5
        networks:
          - fastkafka-network
networks:
    fastkafka-network:
        name: "fastkafka-network"

This configuration will start a single instance of Zookeeper, single instance of Kafka broker and create a ‘hello’ topic (quite enough for a start). To start the configuration, run:

$ docker-compose -f kafka_demo.yaml up -d --wait

This will start the necessary containers and wait till they report that they are Healthy. After the command finishes, you are good to go to try out the FastKafkaAPI capabilities! 🎊

Running the code

After installing FastKafkaAPI and initialising the Kafka broker you can proceed to the ‘First Steps’ part of the tutorial. There, you will write your first Kafka client and producer apps, run them, and interact with them.

You are highly encouraged to follow along the tutorials not just by reading trough them but by implementing the code examples in your own environment. This will not only help you remember the use cases better but also, hopefully, demonstrate to you the ease of use of this library.