Initial Commit

docs/old-network-planning/PHASE_2_IMPLEMENTATION_PLAN.md

@@ -0,0 +1,607 @@
+# Network Library Phase 2: Transport Layer Implementation Plan
+
+Now that we've successfully completed Phase 0 with comprehensive test coverage, we're ready to move on to Phase 2: Transport Layer. This phase will build upon our foundational architecture to implement the transport capabilities needed for actual network communication.
+
+## Overview
+
+Phase 2 focuses on creating the transport abstraction and plugin framework that will allow different transport mechanisms (WebRTC, WebSocket, etc.) to be plugged into the network system. We'll start with a memory-based transport for testing and simulation.
+
+## Implementation Goals
+
+1. **Transport Interface Layer**: Create the transport abstraction interfaces
+2. **Base Transport Class**: Develop an abstract base class for common functionality
+3. **Memory Transport**: Implement an in-memory transport for testing
+4. **Transport Manager**: Create a system to manage multiple transport types
+5. **Plugin Framework**: Build a plugin system for transport extensibility
+6. **Integration**: Connect transport layer with existing NetworkNode implementation
+
+## Detailed Implementation Plan
+
+### 1. Transport Interfaces and Types
+
+First, we'll define the transport interfaces and types that will be used throughout the system:
+
+```typescript
+// src/transport/types.ts
+export interface ConnectionOptions {
+  timeout?: number;
+  retries?: number;
+  metadata?: Record<string, any>;
+}
+
+export interface TransportStats {
+  latency: number;
+  packetLoss: number;
+  bandwidth: number;
+  // Additional stats
+}
+
+export enum TransportState {
+  DISCONNECTED = 'disconnected',
+  CONNECTING = 'connecting',
+  CONNECTED = 'connected',
+  DISCONNECTING = 'disconnecting',
+  ERROR = 'error'
+}
+
+export interface TransportCapabilities {
+  supportsReliableDelivery: boolean;
+  supportsOrdering: boolean;
+  supportsBroadcast: boolean;
+  supportsMulticast: boolean;
+  // Additional capabilities
+}
+
+export interface ITransport {
+  readonly id: string;
+  readonly state: TransportState;
+  readonly capabilities: TransportCapabilities;
+  
+  connect(options?: ConnectionOptions): Promise<void>;
+  disconnect(): Promise<void>;
+  sendTo(recipient: string, message: any): Promise<void>;
+  getStats(): TransportStats;
+  
+  on(event: string, handler: Function): void;
+  off(event: string, handler?: Function): void;
+}
+
+export interface ITransportPlugin {
+  readonly id: string;
+  readonly version: string;
+  readonly capabilities: TransportCapabilities;
+  
+  initialize(options: any): Promise<void>;
+  createTransport(nodeId: string, config: any): ITransport;
+  
+  // Lifecycle hooks
+  onRegister(): void;
+  onUnregister(): void;
+}
+```
+
+### 2. Base Transport Abstract Class
+
+Next, we'll implement an abstract base class that provides common functionality for transport implementations:
+
+```typescript
+// src/transport/base-transport.ts
+export abstract class BaseTransport implements ITransport {
+  protected _id: string;
+  protected _state: TransportState = TransportState.DISCONNECTED;
+  protected _eventEmitter: INetworkEventEmitter;
+  
+  constructor(id: string, eventEmitter?: INetworkEventEmitter) {
+    this._id = id;
+    this._eventEmitter = eventEmitter || new NetworkEventEmitter();
+  }
+  
+  get id(): string {
+    return this._id;
+  }
+  
+  get state(): TransportState {
+    return this._state;
+  }
+  
+  abstract get capabilities(): TransportCapabilities;
+  
+  abstract connect(options?: ConnectionOptions): Promise<void>;
+  abstract disconnect(): Promise<void>;
+  abstract sendTo(recipient: string, message: any): Promise<void>;
+  abstract getStats(): TransportStats;
+  
+  on(event: string, handler: Function): void {
+    this._eventEmitter.on(event, handler);
+  }
+  
+  off(event: string, handler?: Function): void {
+    this._eventEmitter.off(event, handler);
+  }
+  
+  protected _setState(state: TransportState): void {
+    this._state = state;
+    this._eventEmitter.emit(TransportEvents.STATE_CHANGED, this, state);
+  }
+}
+```
+
+### 3. Memory Transport Implementation
+
+We'll implement an in-memory transport for testing and simulation:
+
+```typescript
+// src/transport/memory-transport.ts
+export class MemoryTransport extends BaseTransport {
+  private static _instances: Map<string, MemoryTransport> = new Map();
+  private _messageQueue: Map<string, Queue<any>> = new Map();
+  private _connectionStats: TransportStats;
+  private _simOptions: MemoryTransportSimulationOptions;
+  
+  constructor(id: string, options?: MemoryTransportOptions) {
+    super(id, options?.eventEmitter);
+    this._simOptions = options?.simulationOptions || {};
+    this._connectionStats = this._createDefaultStats();
+    
+    // Register instance globally so other memory transports can communicate
+    MemoryTransport._instances.set(id, this);
+  }
+  
+  get capabilities(): TransportCapabilities {
+    return {
+      supportsReliableDelivery: !this._simOptions.packetLoss,
+      supportsOrdering: true,
+      supportsBroadcast: true,
+      supportsMulticast: true
+    };
+  }
+  
+  async connect(options?: ConnectionOptions): Promise<void> {
+    if (this._state === TransportState.CONNECTED) {
+      return;
+    }
+    
+    this._setState(TransportState.CONNECTING);
+    
+    // Simulate connection delay
+    if (this._simOptions.connectionDelay) {
+      await new Promise(resolve => 
+        setTimeout(resolve, this._simOptions.connectionDelay));
+    }
+    
+    // Simulate connection errors
+    if (this._simOptions.connectionErrorRate && 
+        Math.random() < this._simOptions.connectionErrorRate) {
+      this._setState(TransportState.ERROR);
+      throw new Error('Simulated connection error');
+    }
+    
+    this._setState(TransportState.CONNECTED);
+  }
+  
+  async disconnect(): Promise<void> {
+    if (this._state === TransportState.DISCONNECTED) {
+      return;
+    }
+    
+    this._setState(TransportState.DISCONNECTING);
+    
+    // Simulate disconnection delay
+    if (this._simOptions.disconnectionDelay) {
+      await new Promise(resolve => 
+        setTimeout(resolve, this._simOptions.disconnectionDelay));
+    }
+    
+    this._setState(TransportState.DISCONNECTED);
+    
+    // Cleanup
+    MemoryTransport._instances.delete(this._id);
+  }
+  
+  async sendTo(recipient: string, message: any): Promise<void> {
+    if (this._state !== TransportState.CONNECTED) {
+      throw new Error('Cannot send message: transport not connected');
+    }
+    
+    // Find recipient instance
+    const recipientTransport = MemoryTransport._instances.get(recipient);
+    if (!recipientTransport) {
+      throw new Error(`Recipient not found: ${recipient}`);
+    }
+    
+    // Clone message to simulate network isolation
+    const messageClone = JSON.parse(JSON.stringify(message));
+    
+    // Simulate network conditions
+    if (this._simOptions.latency) {
+      await new Promise(resolve => 
+        setTimeout(resolve, this._simOptions.latency));
+    }
+    
+    // Simulate packet loss
+    if (this._simOptions.packetLoss && 
+        Math.random() < this._simOptions.packetLoss) {
+      // Message lost, but don't throw - silently drop
+      return;
+    }
+    
+    // Deliver message
+    recipientTransport._receiveMessage(this._id, messageClone);
+  }
+  
+  getStats(): TransportStats {
+    return { ...this._connectionStats };
+  }
+  
+  // Additional methods for simulation control
+  setSimulationOptions(options: Partial<MemoryTransportSimulationOptions>): void {
+    this._simOptions = { ...this._simOptions, ...options };
+  }
+  
+  private _receiveMessage(sender: string, message: any): void {
+    this._eventEmitter.emit(TransportEvents.MESSAGE_RECEIVED, message, sender);
+  }
+  
+  private _createDefaultStats(): TransportStats {
+    return {
+      latency: this._simOptions.latency || 0,
+      packetLoss: this._simOptions.packetLoss || 0,
+      bandwidth: Number.POSITIVE_INFINITY
+    };
+  }
+}
+```
+
+### 4. Transport Manager Implementation
+
+Next, we'll implement the TransportManager to handle multiple transport types:
+
+```typescript
+// src/transport/transport-manager.ts
+export class TransportManager {
+  private _transports: Map<string, ITransport> = new Map();
+  private _plugins: Map<string, ITransportPlugin> = new Map();
+  private _config: TransportManagerConfig;
+  private _eventEmitter: INetworkEventEmitter;
+  
+  constructor(config?: Partial<TransportManagerConfig>, eventEmitter?: INetworkEventEmitter) {
+    this._config = { ...this._getDefaultConfig(), ...config };
+    this._eventEmitter = eventEmitter || new NetworkEventEmitter();
+  }
+  
+  registerTransport(transport: ITransport): boolean {
+    if (this._transports.has(transport.id)) {
+      return false;
+    }
+    
+    this._transports.set(transport.id, transport);
+    this._eventEmitter.emit(TransportEvents.TRANSPORT_REGISTERED, transport);
+    return true;
+  }
+  
+  removeTransport(transportId: string): boolean {
+    const transport = this._transports.get(transportId);
+    if (!transport) {
+      return false;
+    }
+    
+    this._transports.delete(transportId);
+    this._eventEmitter.emit(TransportEvents.TRANSPORT_UNREGISTERED, transport);
+    return true;
+  }
+  
+  getTransport(transportId: string): ITransport | undefined {
+    return this._transports.get(transportId);
+  }
+  
+  registerPlugin(plugin: ITransportPlugin): boolean {
+    if (this._plugins.has(plugin.id)) {
+      return false;
+    }
+    
+    this._plugins.set(plugin.id, plugin);
+    plugin.onRegister();
+    this._eventEmitter.emit(TransportEvents.PLUGIN_REGISTERED, plugin);
+    return true;
+  }
+  
+  removePlugin(pluginId: string): boolean {
+    const plugin = this._plugins.get(pluginId);
+    if (!plugin) {
+      return false;
+    }
+    
+    this._plugins.delete(pluginId);
+    plugin.onUnregister();
+    this._eventEmitter.emit(TransportEvents.PLUGIN_UNREGISTERED, plugin);
+    return true;
+  }
+  
+  getPlugin(pluginId: string): ITransportPlugin | undefined {
+    return this._plugins.get(pluginId);
+  }
+  
+  createTransport(nodeId: string, pluginId: string, config?: any): ITransport | null {
+    const plugin = this._plugins.get(pluginId);
+    if (!plugin) {
+      return null;
+    }
+    
+    try {
+      const transport = plugin.createTransport(nodeId, config);
+      this.registerTransport(transport);
+      return transport;
+    } catch (error) {
+      this._eventEmitter.emit(TransportEvents.TRANSPORT_ERROR, error);
+      return null;
+    }
+  }
+  
+  private _getDefaultConfig(): TransportManagerConfig {
+    return {
+      transportFallbackOrder: [],
+      defaultTransportOptions: {}
+    };
+  }
+}
+```
+
+### 5. Transport Plugin Development Kit (PDK)
+
+Now, we'll create a plugin development kit to make it easier to develop transport plugins:
+
+```typescript
+// src/transport/pdk/base-transport-plugin.ts
+export abstract class BaseTransportPlugin implements ITransportPlugin {
+  readonly id: string;
+  readonly version: string;
+  protected _initialized: boolean = false;
+  
+  constructor(id: string, version: string) {
+    this.id = id;
+    this.version = version;
+  }
+  
+  abstract get capabilities(): TransportCapabilities;
+  
+  abstract createTransport(nodeId: string, config: any): ITransport;
+  
+  async initialize(options: any): Promise<void> {
+    // Base implementation
+    this._initialized = true;
+  }
+  
+  onRegister(): void {
+    // Base implementation
+  }
+  
+  onUnregister(): void {
+    // Base implementation
+  }
+}
+
+// src/transport/pdk/transport-testing.ts
+export class TransportTester {
+  static async testBasicFunctionality(transport: ITransport): Promise<boolean> {
+    // Test connection
+    let success = true;
+    
+    try {
+      await transport.connect();
+      success = success && transport.state === TransportState.CONNECTED;
+      
+      // Test disconnection
+      await transport.disconnect();
+      success = success && transport.state === TransportState.DISCONNECTED;
+    } catch (error) {
+      return false;
+    }
+    
+    return success;
+  }
+  
+  static async testReliability(
+    sender: ITransport, 
+    receiver: ITransport, 
+    messageCount: number = 100
+  ): Promise<number> {
+    // Connect both transports
+    await sender.connect();
+    await receiver.connect();
+    
+    // Track received messages
+    let receivedCount = 0;
+    const handler = () => receivedCount++;
+    receiver.on(TransportEvents.MESSAGE_RECEIVED, handler);
+    
+    // Send messages
+    for (let i = 0; i < messageCount; i++) {
+      await sender.sendTo(receiver.id, { test: true, sequence: i });
+    }
+    
+    // Wait for possible delayed messages
+    await new Promise(resolve => setTimeout(resolve, 1000));
+    
+    // Cleanup
+    receiver.off(TransportEvents.MESSAGE_RECEIVED, handler);
+    await sender.disconnect();
+    await receiver.disconnect();
+    
+    // Return reliability ratio (0-1)
+    return receivedCount / messageCount;
+  }
+}
+```
+
+### 6. Memory Transport Plugin
+
+We'll create a plugin for our memory transport:
+
+```typescript
+// src/transport/plugins/memory-transport-plugin.ts
+export class MemoryTransportPlugin extends BaseTransportPlugin {
+  constructor() {
+    super('memory-transport', '1.0.0');
+  }
+  
+  get capabilities(): TransportCapabilities {
+    return {
+      supportsReliableDelivery: true,
+      supportsOrdering: true,
+      supportsBroadcast: true,
+      supportsMulticast: true
+    };
+  }
+  
+  createTransport(nodeId: string, config: any): ITransport {
+    return new MemoryTransport(nodeId, config);
+  }
+}
+```
+
+### 7. Transport Events
+
+We'll add transport-specific events:
+
+```typescript
+// src/transport/events.ts
+export enum TransportEvents {
+  // Connection events
+  CONNECTING = 'transport:connecting',
+  CONNECTED = 'transport:connected',
+  DISCONNECTING = 'transport:disconnecting',
+  DISCONNECTED = 'transport:disconnected',
+  CONNECTION_ERROR = 'transport:connection-error',
+  
+  // Message events
+  MESSAGE_SENT = 'transport:message-sent',
+  MESSAGE_RECEIVED = 'transport:message-received',
+  MESSAGE_ERROR = 'transport:message-error',
+  
+  // Transport management events
+  STATE_CHANGED = 'transport:state-changed',
+  TRANSPORT_REGISTERED = 'transport:registered',
+  TRANSPORT_UNREGISTERED = 'transport:unregistered',
+  
+  // Plugin events
+  PLUGIN_REGISTERED = 'transport:plugin-registered',
+  PLUGIN_UNREGISTERED = 'transport:plugin-unregistered',
+  
+  // Error events
+  TRANSPORT_ERROR = 'transport:error'
+}
+```
+
+### 8. Integrate with NetworkNode
+
+Finally, we'll update the NetworkNode class to use the transport layer:
+
+```typescript
+// Updates to NetworkNode class
+class NetworkNode {
+  // ... existing implementation
+  
+  // Update constructor to accept transport
+  constructor(
+    id: string,
+    level: number,
+    capabilities: NodeCapabilities,
+    transport?: ITransport,
+    config?: NetworkSystemConfig,
+    eventEmitter?: INetworkEventEmitter
+  ) {
+    // ... existing implementation
+    
+    // Set transport if provided
+    if (transport) {
+      this._transport = transport;
+      this._setupTransportEvents();
+    }
+  }
+  
+  // Add method to set transport
+  setTransport(transport: ITransport): void {
+    if (this._transport) {
+      this._teardownTransportEvents();
+    }
+    
+    this._transport = transport;
+    this._setupTransportEvents();
+  }
+  
+  // Update sendMessage to use transport
+  async sendMessage(message: NetworkMessage): Promise<void> {
+    if (!this._transport || this._transport.state !== TransportState.CONNECTED) {
+      throw new Error('Cannot send message: transport not connected or unavailable');
+    }
+    
+    // Emit event for monitoring
+    this._events.emit(NetworkEvents.MESSAGE_SENT, message, this);
+    
+    // Send via transport
+    if (message.targetId) {
+      await this._transport.sendTo(message.targetId, message);
+    } else {
+      // Handle broadcast or hierarchical routing in later phases
+      await this._routeMessage(message);
+    }
+  }
+  
+  // Add private methods for transport events
+  private _setupTransportEvents(): void {
+    if (!this._transport) return;
+    
+    this._transport.on(TransportEvents.CONNECTED, () => {
+      this._state = NodeState.CONNECTED;
+      this._events.emit(NetworkEvents.NODE_CONNECTED, this);
+    });
+    
+    this._transport.on(TransportEvents.DISCONNECTED, () => {
+      this._state = NodeState.DISCONNECTED;
+      this._events.emit(NetworkEvents.NODE_DISCONNECTED, this);
+    });
+    
+    this._transport.on(TransportEvents.MESSAGE_RECEIVED, (message, senderId) => {
+      this._events.emit(NetworkEvents.MESSAGE_RECEIVED, message, this);
+      this._handleIncomingMessage(message);
+    });
+  }
+  
+  private _teardownTransportEvents(): void {
+    if (!this._transport) return;
+    
+    this._transport.off(TransportEvents.CONNECTED);
+    this._transport.off(TransportEvents.DISCONNECTED);
+    this._transport.off(TransportEvents.MESSAGE_RECEIVED);
+  }
+}
+```
+
+## Testing Plan
+
+1. **Unit Tests**:
+   - Test ITransport interface implementations
+   - Test MemoryTransport functionality
+   - Test TransportManager operations
+   - Test transport plugins
+
+2. **Integration Tests**:
+   - Test communication between nodes using transports
+   - Test transport fallback strategies
+   - Test plugin discovery and initialization
+
+3. **Performance Tests**:
+   - Test message throughput
+   - Test latency simulation
+   - Test reliability under packet loss
+
+## Implementation Schedule
+
+1. **Week 1**: Implement transport interfaces and base classes
+2. **Week 2**: Implement MemoryTransport and TransportManager
+3. **Week 3**: Create transport plugin framework
+4. **Week 4**: Integrate with NetworkNode and write tests
+
+## Conclusion
+
+This plan outlines a comprehensive approach to implementing the Transport Layer (Phase 2) of our network library. By focusing on a plugin-based architecture, we enable easy extension to different transport mechanisms while providing a solid in-memory implementation for testing and simulation.