Skip to content

Commit

Permalink
Revamp seigr_protocol Proto Definitions for Decentralized Data Structure
Browse files Browse the repository at this point in the history
Summary:

Created and Updated Proto Files: Refined existing .proto files and added new structures to enhance data modularity, privacy options, and support for Seigr’s decentralized, traceable data ecosystem.
Naming Consistency: Ensured naming conventions across enums and message types align with Seigr's modular architecture, mitigating naming conflicts and improving readability.
Protocol Enhancements:
Alias & Privacy in identity.proto: Expanded support for alias identities with configurable privacy levels and flexible linkage back to main Seigr IDs, enabling creators to manage public personas while keeping ownership traceable.
Resource Management & Replication Flexibility: Updated replication.proto, resource_management.proto, and common_requests.proto with refined enums for ReplicationPriority, ReplicationTrigger, and ReplicationStrategy. Added resource state and usage controls.
Temporal and Lineage Tracking: Consolidated temporal snapshot management in temporal.proto and lineage records in lineage.proto to support versioning, rollback, and historical data tracking.
Error & Script Handling: Structured error_handling.proto and script.proto to manage error resilience and script permissions, enabling seamless fallback strategies and controlled execution environments.
Optimized Imports: Streamlined .proto file imports and standardized package dependencies to avoid redundancy, ensuring a modular but interconnected protocol design.
Decoupled Identity and User Management: Reorganized identity.proto and user_management.proto to decouple identity verification and account control from user roles and preferences, allowing for independent scaling and modular use in authentication.
Implications:

Codebase Impact: This update necessitates revision of imports and protocol references across the codebase. Naming consistency and alias support enhance data privacy and access control capabilities, supporting scalable privacy options for users.
Testing & Validation: Expanded .proto definitions require integration with updated test cases. The refactored enums and messages will need validation to confirm compatibility with Seigr’s evolving access, lineage, and replication management features.
Documentation: Updated documentation needed across seigr_protocol to outline new message types, privacy configurations, and protocol functions.
  • Loading branch information
sergism77 committed Nov 11, 2024
1 parent 1a6d7f4 commit 25d9954
Show file tree
Hide file tree
Showing 76 changed files with 3,533 additions and 493 deletions.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Seigr is a modular, eco-inspired file and network ecosystem designed for resilie

## Overview

Seigr offers a unique, mycelium-inspired digital ecosystem where files, nodes, and network layers work in harmony. The aim is to create a modular system where each component adapts dynamically to network conditions and user needs, contributing to the resilience and sustainability of the network as a whole. The platform’s modular architecture includes several core components: lineage tracking, replication management, identity services, and integrity protocols.
Seigr offers a unique, mycelium-inspired digital ecosystem where files, hyphens, and network layers work in harmony. The aim is to create a modular system where each component adapts dynamically to network conditions and user needs, contributing to the resilience and sustainability of the network as a whole. The platform’s modular architecture includes several core components: lineage tracking, replication management, identity services, and integrity protocols.

This README provides a summary of recent work, completed milestones, and a roadmap of planned development.

Expand All @@ -20,7 +20,7 @@ Seigr’s structure consists of the following primary modules:
- **Lineage**: Modular components for tracking file history and lineage integrity.
- **Replication**: Mechanisms to ensure file redundancy, resilience, and self-healing capabilities.
- **File Management**: Adaptive, self-describing file structure that supports efficient replication and validation.
- **Identity**: Decentralized identity management for secure user and node validation.
- **Identity**: Decentralized identity management for secure user and hyphen validation.
- **IPFS Integration**: Leveraging IPFS for distributed file storage.

## Recent Work and Updates
Expand Down Expand Up @@ -53,7 +53,7 @@ The following is a roadmap of completed, ongoing, and future tasks:
| **Replication** | *In Progress* | Modularized replication controller and manager. |
| **File Management**| *Planned* | Adaptive, context-aware file structure. |
| **Integrity Checks**| *In Progress* | Developing integrity protocols for self-healing. |
| **Identity** | *Planned* | Decentralized identity management for nodes. |
| **Identity** | *Planned* | Decentralized identity management for hyphens. |
| **IPFS Integration**| *In Progress* | Integrating with IPFS for distributed storage. |
| **Documentation** | *Ongoing* | Enhanced contributor and sponsor guidance. |

Expand Down
58 changes: 41 additions & 17 deletions project_tree.txt
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,11 @@
│   │   ├── dot_seigr.py
│   │   ├── file_format
│   │   │   ├── __init__.py
│   │   │   ├── decoder.py
│   │   │   ├── encoder.py
│   │   │   ├── interpreter.py
│   │   │   ├── manager.py
│   │   │   ├── metadata.py
│   │   │   ├── seigr_coordinate.py
│   │   │   ├── seigr_integrity.py
│   │   │   ├── seigr_integrity_manager.py
Expand Down Expand Up @@ -70,22 +75,6 @@
│   │   ├── seigr_constants.py
│   │   ├── seigr_file.py
│   │   ├── seigr_ipfs_utils.py
│   │   ├── seigr_protocol
│   │   │   ├── __init__.py
│   │   │   ├── __pycache__
│   │   │   │   ├── __init__.cpython-310.pyc
│   │   │   │   ├── lineage_pb2.cpython-310.pyc
│   │   │   │   ├── manager.cpython-310.pyc
│   │   │   │   └── seed_dot_seigr_pb2.cpython-310.pyc
│   │   │   ├── decoder.py
│   │   │   ├── encoder.py
│   │   │   ├── interpreter.py
│   │   │   ├── lineage.proto
│   │   │   ├── lineage_pb2.py
│   │   │   ├── manager.py
│   │   │   ├── metadata.py
│   │   │   ├── seed_dot_seigr.proto
│   │   │   └── seed_dot_seigr_pb2.py
│   │   ├── serialization_manager.py
│   │   └── temporal_layer_manager.py
│   ├── identity
Expand Down Expand Up @@ -137,6 +126,41 @@
│   │   ├── replication_self_heal.py
│   │   ├── replication_threat.py
│   │   └── target_calculator.py
│   ├── seigr_protocol
│   │   ├── __init__.py
│   │   ├── __pycache__
│   │   │   ├── __init__.cpython-310.pyc
│   │   │   ├── lineage_pb2.cpython-310.pyc
│   │   │   ├── manager.cpython-310.pyc
│   │   │   └── seed_dot_seigr_pb2.cpython-310.pyc
│   │   ├── access_control.proto
│   │   ├── access_policies.proto
│   │   ├── analytics.proto
│   │   ├── audit_logging.proto
│   │   ├── common.proto
│   │   ├── compiled
│   │   ├── coordinate.proto
│   │   ├── encryption.proto
│   │   ├── error_handling.proto
│   │   ├── file_metadata.proto
│   │   ├── hashing.proto
│   │   ├── identity.proto
│   │   ├── integrity.proto
│   │   ├── interpretation.proto
│   │   ├── lineage.proto
│   │   ├── monitoring.proto
│   │   ├── network_management.proto
│   │   ├── notification.proto
│   │   ├── orchestration.proto
│   │   ├── pipeline.proto
│   │   ├── replication.proto
│   │   ├── resource_management.proto
│   │   ├── rollback.proto
│   │   ├── script.proto
│   │   ├── seed_dot_seigr.proto
│   │   ├── segment_metadata.proto
│   │   ├── temporal.proto
│   │   └── user_management.proto
│   ├── static
│   │   ├── css
│   │   │   └── style.css
Expand Down Expand Up @@ -190,4 +214,4 @@
│   └── test_seigr_file.py
└── uploads

40 directories, 151 files
41 directories, 174 files
6 changes: 3 additions & 3 deletions src/dot_seigr/access_control.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ class AccessControlManager:
def __init__(self, creator_id):
self.creator_id = creator_id
self.acl = [{"user_id": "default", "role": "viewer", "permissions": "read"}]
self.access_context = {"access_count": 0, "last_accessed": "", "node_access_history": []}
self.access_context = {"access_count": 0, "last_accessed": "", "hyphen_access_history": []}

def record_access(self, node_id):
def record_access(self, hyphen_id):
self.access_context["access_count"] += 1
self.access_context["last_accessed"] = datetime.now(timezone.utc).isoformat()
self.access_context["node_access_history"].append(node_id)
self.access_context["hyphen_access_history"].append(hyphen_id)
2 changes: 1 addition & 1 deletion src/dot_seigr/dot_seigr.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
TriggerEvent,
OperationLog
)
from src.dot_seigr.seigr_protocol.manager import LinkManager
from dot_seigr.file_format.manager import LinkManager

# Setup logging
logger = logging.getLogger(__name__)
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import os
import logging
from src.crypto.encoding_utils import encode_to_senary
from src.dot_seigr.seigr_protocol.manager import SeigrClusterManager
from dot_seigr.file_format.manager import SeigrClusterManager
from src.dot_seigr.seigr_file import SeigrFile
from src.dot_seigr.seigr_constants import TARGET_BINARY_SEGMENT_SIZE

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -155,13 +155,13 @@ def load_and_validate_metadata(self, metadata_file: str) -> FileMetadata:

return metadata

def log_access(self, metadata: FileMetadata, node_id: str):
def log_access(self, metadata: FileMetadata, hyphen_id: str):
"""
Logs access to a .seigr segment or file, updating access logs.
Args:
metadata (FileMetadata): Metadata Protobuf to log access for.
node_id (str): Unique identifier of the accessing node.
hyphen_id (str): Unique identifier of the accessing hyphen.
"""
self.metadata_manager.update_access_log(metadata, node_id)
logger.debug(f"Access logged for node {node_id} in metadata.")
self.metadata_manager.update_access_log(metadata, hyphen_id)
logger.debug(f"Access logged for hyphen {hyphen_id} in metadata.")
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ def __init__(self, creator_id: str, original_filename: str = None, original_exte
self.version = version
self.timestamp = int(datetime.now(timezone.utc).timestamp()) # Cluster creation timestamp
self.cluster_hash = None # To be generated based on segments
self.replication_controller = ReplicationController(min_replication=3, demand_threshold=10, network_nodes=["node1", "node2"])
self.replication_controller = ReplicationController(min_replication=3, demand_threshold=10, network_hyphens=["hyphen1", "hyphen2"])

def add_segment(self, segment_hash: str, index: int, threat_level: int = 0):
"""
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -96,13 +96,13 @@ def load_metadata(self, file_path: str) -> FileMetadata:
logger.error(f"Failed to load metadata from {file_path}: {e}")
raise

def update_access_log(self, metadata: FileMetadata, node_id: str):
def update_access_log(self, metadata: FileMetadata, hyphen_id: str):
if not metadata.HasField("access_context"):
metadata.access_context.CopyFrom(AccessContext(access_count=0, last_accessed="", node_access_history=[]))
metadata.access_context.CopyFrom(AccessContext(access_count=0, last_accessed="", hyphen_access_history=[]))

access_context = metadata.access_context
access_context.access_count += 1
access_context.last_accessed = datetime.now(timezone.utc).isoformat()
access_context.node_access_history.append(node_id)
access_context.hyphen_access_history.append(hyphen_id)

logger.debug(f"Access log updated for node {node_id}. Total access count: {access_context.access_count}")
logger.debug(f"Access log updated for hyphen {hyphen_id}. Total access count: {access_context.access_count}")
26 changes: 0 additions & 26 deletions src/dot_seigr/seigr_protocol/lineage.proto

This file was deleted.

31 changes: 0 additions & 31 deletions src/dot_seigr/seigr_protocol/lineage_pb2.py

This file was deleted.

Loading

0 comments on commit 25d9954

Please sign in to comment.