Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GLCC 2023] 提供分布式锁的能力 / Provide Ability for distributed locks. #10378

Closed
KomachiSion opened this issue Apr 24, 2023 · 5 comments
Labels
Nacos3.0 Nacos 3.0 Architecture Evolution Summer

Comments

@KomachiSion
Copy link
Collaborator

KomachiSion commented Apr 24, 2023

GitLink编程夏令营(GLCC),是在CCF中国计算机学会指导下,由CCF开源发展委员会(CCF ODC)举办的面向全国高校学生的暑期编程活动。活动将覆盖近千所高校,并联合各大开源基金会、开源企业、开源社区、开源专家,旨在鼓励青年学生通过参加真实的开源软件开发,提升自身技术能力,为开源社区输送优秀人才。为青年学生提供开放友好的交流平台,希望进一步推动国内开源社区的繁荣发展。

Nacos将会作为指导组织参加本次的“GitLink编程夏令营(GLCC)”

背景

Nacos 社区向社区用户征集Nacos3.0中期望的功能中,支持分布式锁的功能是呼声最高的功能之一。分布式锁作为分布式应用常用功能,目前主要有Zookeeper或redis产品来实现,许多用户已经使用Nacos替换Zookeeper进行服务管理和配置管理, 但由于Nacos无法支持分布式锁的功能,无法移除Zookeeper的依赖,还需要额外运维Zookeeper集群。如果Nacos能够实现分布式锁功能,能在一定程度帮助用户减少依赖系统,降低微服务应用架构的复杂度。

由于分布式锁的原理和实现较为复杂,同学可以提前学习Zookeeper和Redis等现有产品的分布式锁实现原理和方式,并参考其实现方式进行方案设计,但不可以直接照搬或抄袭。

目标

提供类似Zookeeper或redis的分布式锁的功能;如果有余力可以给予该分布式锁功能添加协助用户服务选主的功能。

难度

进阶

导师

王建伟
786594722@qq.com

产出要求

  • 设计Nacos分布式锁的功能方案,并实现功能
  • 新增Nacos分布式锁功能的使用文档
  • 给予分布式锁功能,实现协助用户服务选主功能(可选)
  • 新增协助用户服务选主功能的使用文档(可选)

能力要求

  • 熟悉Java编程语言
  • 熟悉常见的设计模式
  • 熟悉并发编程
  • 了解Zookeeper和Redis分布式锁的原理和实现
  • 了解CAP理论
  • 了解Paxos,raft等常见CP协议
  • 熟悉markdown

GitLink Programming Summer Camp (GLCC) is a summer programming activity for college students nationwide, under the guidance of the China Computer Federation (CCF) and organized by the CCF Open Source Development Committee (CCF ODC). The event will cover nearly a thousand colleges and universities, and will unite various open source foundations, enterprises, communities, and experts with the aim of encouraging young students to participate in real open source software development, enhancing their technical skills, and providing outstanding talent for the open source community. GLCC also provides an open and friendly platform for young students to communicate and hopes to further promote the prosperity and development of the domestic open source community.

Nacos will participate as a guiding organization in “GitLink Programming Summer Camp (GLCC)”.

Background

The Nacos community is soliciting the desired features for Nacos 3.0 from users. Among these features, Provide Ability for distributed locks is one of the most requested ones. As a common feature in distributed applications, currently, Zookeeper or Redis are mainly used to implement distributed locks. Many users have already replaced Zookeeper with Nacos for service and configuration management, but because Nacos cannot support distributed lock functionality, it cannot remove the Zookeeper dependency and additional maintain Zookeeper cluster. If Nacos can implement distributed lock feature, it can help users reduce system dependencies to some extent and reduce the complexity of microservice application architecture.

Due to the principle and implementation of distributed lock is relatively complex, students can learn the implementation principles and methods of distributed locks in existing products such as Zookeeper and Redis in advance, and refer to their implementation way for program design, but copying or directly using their way is not allowed.

Target

Provide distributed lock functionality similar to Zookeeper or Redis; if possible, add feature to assist users in selecting the leader service for this distributed lock.

Difficulty

Advanced

Mentor

Jianwei Wang
786594722@qq.com

Output Requirements

  • Design Nacos distributed locks and implement the feature
  • Add usage documentation for Nacos distributed lock
  • Provide leader service selection by distributed lock feature (optional)
  • Add usage documentation for the leader service selection (optional)

Technical Requirements

  • Familiar with Java programming language
  • Familiar with common design patterns
  • Familiar with concurrent programming
  • Understand the principles and implementation of distributed locks in Zookeeper and Redis
  • Understand CAP theory
  • Understand common CP protocols such as Paxos and Raft
  • Familiar with writing markdown documents
@azhsmesos
Copy link

  • english

    Hello,I have some ideas about something like zookeeper, regarding the issue mentioned above, if we want to implement a distributed lock without relying on third-party components, and assume that the lock is implemented using an integer, where 1 represents the holder of the lock, and 0 represents the lock being idle, it is essentially a consensus problem. However, in distributed systems, according to FLP theorem, we know that safety and liveness cannot be satisfied simultaneously in an asynchronous network environment. For example, Paxos satisfies the safety property. Since most of our production environments are asynchronous networks, there is no perfect solution for such scenarios, and we need to weigh and choose appropriate solutions based on specific circumstances.

    Currently, the popular approach for implementing distributed locks in the industry is based on ZooKeeper, which creates multiple sequential nodes under the current node when multiple clients apply for one lock. The clients will then lock and unlock according to the sequence, and the unlock operation will notify other clients through ZooKeeper's watch mechanism (by listening to the previous node of the current node, to prevent the herd effect), thus allowing other clients to acquire the lock. Similarly, in Nacos, we can simulate this approach to design distributed locks.

  • 中文
    你好,相对于上面这个issue,我们要不依赖外部第三方组件实现一个分布式锁,假如锁用一个int来实现,1代表拿到锁的holder,0代表锁空闲,那么本质上是一个consensus问题。
    但在分布式系统中,根据FLP定理,我们知道在异步网络的情况下无法同时满足safety和liveness,例如,paxos是safety属性的。而我们的生产环境大部分都是异步网络,故而在异步网络的场景下我们没有一个很完美的实现方式,需要根据具体情况去权衡和选择。
    而当前业界比较流行的分布式锁系统中,基于zookeeper的实现方式是一种比较流行的分布式锁实现方式,其原理在于我们在多个客户端进行一把分布式锁进行加锁时,会在当前节点下面创建多个顺序节点(可以参考当前流行的Curator框架),然后按照顺序进行加锁和解锁,解锁时会通过zookeeper的watch机制通知其他客户端(这儿是当前客户端对当前节点的上个节点进行监听,防止羊群效应),然后客户端进行加锁处理。同理在nacos中,我们也可以模拟这种方式来进行分布式锁的设计.

@lutcoding
Copy link

了解etcd基于mvcc实现的分布式锁源码,但不太会java,有资格参加吗

@li-xiao-shuang
Copy link
Collaborator

li-xiao-shuang commented May 10, 2023

终于搞起来了啊👍🏻👍🏻👍🏻

@scwlkq
Copy link
Contributor

scwlkq commented May 29, 2023

zhgg牛

@KomachiSion KomachiSion changed the title [Summer 2023] 提供分布式锁的能力 / Provide Ability for distributed locks. [GLCC 2023] 提供分布式锁的能力 / Provide Ability for distributed locks. Jun 14, 2023
@KomachiSion
Copy link
Collaborator Author

JianweiWang added a commit that referenced this issue Sep 25, 2023
JianweiWang added a commit that referenced this issue Nov 27, 2023
[GLCC #10378] Improve log and memory lack.
KomachiSion added a commit that referenced this issue Dec 19, 2023
[ISSUE #10378] merge develop into glcc branch
JianweiWang added a commit that referenced this issue Feb 20, 2024
[ISSUE #10378]add lock query count and rt metrics
@KomachiSion KomachiSion added the Nacos3.0 Nacos 3.0 Architecture Evolution label Dec 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Nacos3.0 Nacos 3.0 Architecture Evolution Summer
Projects
None yet
Development

No branches or pull requests

5 participants