NekoQ-Bootstrap is the core and fundamental service for NekoQ related services.
It is responsible for the bootstrap service discovery and basic configuration storage.
In order to make NekoQ-Bootstrap as simple as possible, the usecases are kept as essential ones.
- run
go build
in foldercmd/nekoq-bootstrap
- copy
bootstrap.toml.example
tobootstrap.toml
- put them in the same folder
- run
nekoq-bootstrap
DC1 DC2
┌──────────────────────────────────────────────────┐ ┌──────────────────────────────────────────────────┐
│ │ │ │
│ ┌─────────────────┐ ┌─────────────────┐ │ │ ┌─────────────────┐ ┌─────────────────┐ │
│ │ │ │ ├──┼───┼─►│ │ │ │ │
│ │ NekoQ Bootstrap │◄────────┤ NekoQ Discovery │ │ │ │ NekoQ Discovery ├────────►│ NekoQ Bootstrap │ │
│ │ Cluster │ │ │◄─┼───┼──┤ │ │ Cluster │ │
│ └─────────────────┘◄┐ ┌►└─────────────────┘ │ │ └─────────────────┘◄┐ ┌►└─────────────────┘ │
│ │ │ │ │ │ │ │
│ │ │ │ │ │ │ │
│ │ │ │ │ │ │ │
│ │ │ │ │ │ │ │
│ │ │ │ │ │ │ │
│ ┌─────┴─────┴─────┐ │ │ ┌─────┴─────┴─────┐ │
│ │ │ │ │ │ │ │
│ │ NekoQ Services │ │ │ │ NekoQ Services │ │
│ │ │ │ │ │ │ │
│ └─────────────────┘ │ │ └─────────────────┘ │
│ │ │ │
│ │ │ │
│ │ │ │
└──────────────────────────────────────────────────┘ └──────────────────────────────────────────────────┘
NekoQ-Boostrap is the bootstrap for all nekoq related and user services to discover fundamental services.
It acts as a service discovery but only do basic static and dynamic discovery within one single datacenter.
These services can be registered in nekoq-bootstrap:
- nekoq discovery service(embedded in nekoq)
- NekoQ-Security
- nekoq consistency system(embedded in nekoq)
- And simple storage
The workflow for a service to start is:
- Query nekoq-bootstrap -> discovery + nekoq-security + consistency system + key configurations
- Prepare authentications using nekoq-security
- Find essential services using discovery
- Get essential configurations from consistency system
- Do initialization within the service
- Ready to serve
The design principles:
- Keep NekoQ-Bootstrap as simple as possible in order to get high availability
- Only for several key components
- Easy to configure & start
- Keep as little as possible data to persist
Shared component types should be:
- discovery
- security
- consistency
- batch
- message queue
- agent/service bus
- DNS service discovery: A record
- DNS service discovery: TXT record
- DNS service discovery: SRV record
- DNS upstream name server support
- DNS over http - rfc8484
- DNS over https - rfc8484
- DNS service discovery - AAAA/MX/CNAME
- DNS service discovery - SOA/PTR
- DNS Sec
- DNS TCP
- Recursive DNS
- Authority DNS Server
- DNS caching and TTL
- DNS recursion support
- DNS Authoritative
- Register several types of service
- Support register same node to several NekoQ-Bootstrap. Note: DO NOT use different data in this case. Otherwise only the latest registration will be effect under current HA strategy within the cluster.
- Peer auth
- Peer data sync
- Peer auth
- Peer data sync: dns data
- KV store
- web manager
- Graceful shutdown
- Combine dns module and http module
Use simple replication model:
Copy local services to every node that requesting the data
In this case, when every node in the cluster listens to other nodes, the cluster will reach a consistent state in which every node has the full data set of the cluster.
In addition, one node can be easily configured to observer mode when it listens to the cluster but nobody else listens to itself.
However, the drawbacks of this design is:
- If network splits, it can cause brain split as no consistency protocol runs to guarantee the majority.
- Data sync may great impact the network infrastructure even when full data sync happens.
- Keep as minimum dependencies as possible
- Refactor HA module
- Web manager
- DNS module: dns/http for A record
- Http module: query/register service
- HA module: data sync