Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Output the aggregate result to TiDB #324

Open
wants to merge 1 commit into
base: 5.4.1-post
Choose a base branch
from

Conversation

Ishiihara
Copy link

This is a POC to result to TiDB.

Here are the steps to run the example:

  1. Follow the instructions in tiup.io to deploy a TiDB cluster.
  2. Connect to TiDB
    $mysql --host 127.0.0.1 --port 4000 -u root
  3. Create a table under the test database:
    mysql>USE test;
    mysql>CREATE TABLE wordcount (word VARCHAR(255) NOT NULL, wordcount BIGINT, PRIMARY KEY (word));
  4. Download Kafka, unzip it and in the Kafka directory, start Zookeeper and Kafka
    $bin/zookeeper-server-start.sh config/zookeeper.properties
    $bin/kafka-server-start.sh config/server.properties
  5. in the Kafka directory, create topics:
    $bin/kafka-topics.sh--create --topic streams-plaintext-input --zookeeper localhost:2181 --partitions 1 --replication-factor 1
    $bin/kafka-topics --create --topic streams-wordcount-output --zookeeper localhost:2181 --partitions 1 --replication-factor 1
  6. In the kafka-streams-examples directory, build and package the application
    $ mvn -DskipTests=true clean package
  7. In the kafka-streams-examples directory, run the application
    $java -cp target/kafka-streams-examples-5.4.1-standalone.jar \ io.confluent.examples.streams.WordCountLambdaExample
  8. In the Kafka directory, use the console producer to produce to the input topic
    $bin/kafka-topics.sh --create --topic streams-plaintext-input \ --zookeeper localhost:2181 --partitions 1 --replication-factor 1
  9. Inspect the result
    mysql> SELECT * from wordcount;

@ghost
Copy link

ghost commented Mar 9, 2020

It looks like @Ishiihara hasn't signed our Contributor License Agreement, yet.

The purpose of a CLA is to ensure that the guardian of a project's outputs has the necessary ownership or grants of rights over all contributions to allow them to distribute under the chosen licence.
Wikipedia

You can read and sign our full Contributor License Agreement here.

Once you've signed reply with [clabot:check] to prove it.

Appreciation of efforts,

clabot

@abbccdda
Copy link

Thanks for the PR. But I don't think this has anything to do with TiDB? Any MySQL client could run with it?

@OneCricketeer
Copy link

Agreed. It'd be best to use Kafka Connect JDBC sink for such a use case

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants