in ,

ThreeDotsLabs / watermill, Hacker News

ThreeDotsLabs / watermill, Hacker News


                    

        

CircleCIGo Report Cardcodecov

Watermill is a Go library for working efficiently with message streams. It is intended for building event driven applications, enabling event sourcing, RPC over messages, sagas and basically whatever else comes to your mind. You can use conventional pub / sub implementations like Kafka or RabbitMQ, but also HTTP or MySQL binlog if that fits your use case.

Goals

  • Easyto understand.
  • Universal– event-driven architecture, messaging, stream processing, CQRS – use it for whatever you need.
  • Fast(seeBenchmarks).
  • Flexiblewith middlewares, plugins and Pub / Sub configurations.
  • Resilient– using proven technologies and passing stress tests (seeStability).

Getting Started

Pick what you like the best or see in order:

  1. Follow theGetting Started guide.
  2. See examples below.
  3. Read the full documentation:https://watermill.io/

Examples

Background

Building distributed and scalable services is rarely as easy as some may suggest. There is a lot of hidden knowledge that comes with writing such systems. Just like you don’t need to know the whole TCP stack to create a HTTP REST server, you shouldn’t need to study all of this knowledge to start with building message-driven applications.

Watermill’s goal is to make communication with messages as easy to use as HTTP routers. It provides the tools needed to begin working with event-driven architecture and allows you to learn the details on the go.

At the heart of Watermill there is one simple interface:

FUNCMessage)([] *Message,error)

Your handler receives a message and decides whether to publish new message (s) or return an error. What happens next is up to the middlewares you’ve chosen.

You can find more about our motivations in ourIntroducing Watermill (blog post) .

Pub / Subs

All publishers and subscribers have to implement an interface:

typePublisherinterface{Publish(topic  (string) , messages ... * Message)error(Close)  ()error}typeSubscriberinterface{Subscribe(ctx context.  (Context) , topicstring) ((chan)  * Message,error)  (Close)  ()error}

Supported Pub / Subs:

All Pub / Subs implementation documentation can be found in Thedocumentation.

Contributing

Please check ourcontributing guide.

Stability

Watermill v1.0.0 has been released and is production-ready. The public API is stable and will not change without changing the major version.

To ensure that all Pub / Subs are stable and safe to use in production, we created aset of teststhat need to pass for each of the implementations before merging to master. All tests are also executed instressmode – that means that we are running all the tests20 xin parallel.

All tests are run with the race condition detector enabled (- raceflag in tests

Benchmarks

Initial tools for benchmarking Pub / Subs can be found in (watermill-benchmark) .

All benchmarks are being done on a single 16 CPU VM instance, running one binary and dependencies in Docker Compose.

These numbers are meant to serve as a rough estimate of how fast messages can be processed by different Pub / Subs. Keep in mind that the results can be vastly different, depending on the setup and configuration (both much lower and higher).

Here’s the short version for message size of 16 bytes.

(Pub / Sub)

Publish (messages / s) Subscribe (messages / s)
Kafka (one node) 63506 110811
Kafka (5 nodes) 70252 117529
NATS 76208 38169
SQL (MySQL) 6989 143
Google Cloud Pub / Sub 7416 39591
AMQP 2408 10608
GoChannel 272938 101371

Support

If you didn’t find the answer to your question inthe documentation, feel free to ask us directly!

Please join us on the#WATERMILLchannel on theGophers slack: You can get an invite (here) .

Every bit of feedback is very welcome and appreciated. Please submit it usingthe survey.

Why the name?

It processes streams!

License

MIT License

  

Brave Browser
Read More
Payeer

What do you think?

Leave a Reply

Your email address will not be published. Required fields are marked *

GIPHY App Key not set. Please check settings

HTTP Toolkit, Hacker News

HTTP Toolkit, Hacker News

Baby Archie Wore A Royally Adorable Outfit In South Africa – HuffPost, Huffpost.com

Baby Archie Wore A Royally Adorable Outfit In South Africa – HuffPost, Huffpost.com