suaveplan/docs/old-network-planning/test-failures-analysis.md

Analysis of Test Failures in Network Library

Failing Tests Overview

When running bun test libs/network/src, these tests fail:

✗ MemoryTransport > Simulated network conditions > should simulate connection delay
✗ MemoryTransport > Simulated network conditions > should simulate latency
✗ VotingManager > session management > should get vote history
✗ NetworkNodePhase4 > initialization > should register core network reliability calculator
✗ NetworkNodePhase4 > host selection > should select the best host
✗ MessageBatch > Age tracking > should track batch age
✗ TransportTester > testMessageDelivery > should return true when message is delivered
✗ TransportTester > testReliability > should report perfect reliability when all messages delivered
✗ TransportTester > testReliability > should report partial reliability when some messages lost

However, when running each test file individually, all tests pass. This indicates an issue with test isolation.

Root Cause Analysis for Each Failing Test

1. MemoryTransport - Connection Delay & Latency Tests

File: libs/network/src/transport/memory-transport.spec.ts

Issues:

• Both tests use real setTimeout for time-based assertions
• The MemoryTransport class uses a static _instances Map that isn't properly reset between tests
• Tests make timing assertions that can be affected by other running tests

Relevant code:

it('should simulate connection delay', async () => {
  const simulationOptions: MemoryTransportSimulationOptions = {
    connectionDelay: 50
  };

  const transport = new MemoryTransport('delayed-transport', {
    simulationOptions,
    eventEmitter: new NetworkEventEmitter()
  });

  const startTime = Date.now();
  await transport.connect();
  const endTime = Date.now();

  expect(endTime - startTime).toBeGreaterThanOrEqual(50);
});

Solution:

1. Update MemoryTransport to accept a TimerController instance
2. Add a static resetRegistry method to clear all instances
3. Refactor tests to use TimerController for time simulation

2. VotingManager - Vote History Test

File: libs/network/src/voting/voting-manager.spec.ts

Issues:

• The test creates and completes multiple sessions
• Some sessions may be added to history out of order due to timing issues
• Depends on proper cleanup between tests

Relevant code:

it('should get vote history', () => {
  // Create a vote session
  const sessionId = votingManager.createVoteSession(VoteType.HOST_SELECTION, 'session');

  // Complete the session to move it to history
  const session = votingManager.getSession(sessionId);
  if (session) {
    // Add some votes
    (session as any)._testAddVote({
      voteId: 'vote1',
      voterId: 'node1',
      value: { selectedHost: 'node1' },
      timestamp: Date.now()
    });

    // Force completion
    (session as any)._testSetStatus(VoteSessionStatus.COMPLETED);
  }

  // Get vote history
  const history = votingManager.getVoteHistory();

  // Verify history
  expect(history.length).toBe(1);
  expect(history[0].id).toBe(sessionId);
});

Solution:

1. Ensure VotingManager properly cleans up between tests
2. Use a consistent timestamp source that can be controlled
3. Improve assertion to check if the expected session is in the history, not just the count

3. NetworkNodePhase4 - Initialization and Host Selection

File: libs/network/src/node/network-node-phase4.spec.ts

Issues:

• The test creates multiple extensions in the same test file
• Each extension registers calculators with shared resources
• The host selection test depends on mocked responses that could be affected by other tests

Relevant code:

it('should register core network reliability calculator', () => {
  // Spy on metrics manager
  const registerCalculatorSpy = jest.spyOn(nodePhase4.metricsManager, 'registerCalculator');

  // Create a new extension to verify registration
  const newExtension = new NetworkNodePhase4(mockNode, eventEmitter);

  // Verify calculator was registered
  expect(registerCalculatorSpy).toHaveBeenCalled();
  
  // Clean up
  newExtension.dispose();
});

Solution:

1. Ensure proper cleanup between tests with more thorough dispose() methods
2. Isolate dependencies so that new instances don't interfere with each other
3. Replace event emitter instances in each test

4. MessageBatch - Age Tracking

File: libs/network/src/router/message-batch.spec.ts

Issues:

• Uses real setTimeout which can be affected by other tests
• Depends on actual timing for assertions

Relevant code:

it('should track batch age', async () => {
  // Create a batch and wait a bit
  const initialAge = batch.age;

  // Wait at least 10ms
  await new Promise(resolve => setTimeout(resolve, 10));

  // Age should have increased
  expect(batch.age).toBeGreaterThan(initialAge);
  expect(batch.age).toBeGreaterThanOrEqual(10);
});

Solution:

1. Update MessageBatch to optionally accept a TimerController
2. Refactor test to use controlled time advancement

5. TransportTester - Message Delivery and Reliability Tests

File: libs/network/src/transport/pdk/transport-testing.spec.ts

Issues:

• Tests rely on asynchronous callbacks and real timing
• MockTransport implementation uses real setTimeout
• Race conditions can cause tests to fail when run concurrently

Relevant code:

it('should return true when message is delivered', async () => {
  // Setup receiver to simulate receiving the message
  const testMessage = { test: true };

  // We need to simulate the receipt of the message when the sender sends it
  sender.on(TransportEvents.MESSAGE_SENT, (msg) => {
    receiver.simulateMessageReceived(msg, sender.id);
  });

  const result = await TransportTester.testMessageDelivery(sender, receiver, testMessage);
  expect(result).toBe(true);
});

Solution:

1. Update MockTransport to use TimerController
2. Modify TransportTester to optionally accept a TimerController
3. Use controlled time advancement rather than actual timeouts

Common Patterns and Solution Strategy

Core Issues

1. Shared Static State: Several classes use static properties or singletons
2. Real Timing Dependency: Tests rely on actual setTimeout/setInterval
3. Timestamp Variations: Inconsistent approaches to time tracking

Solution Strategy

1. Refactor TimerController:

   • Remove the singleton pattern
   • Create factory for isolated instances
   • Ensure proper cleanup methods

2. Update Key Classes:

   interface TimeOptions {
     timerController?: TimerController;
     useRealTime?: boolean; // Default to false in tests
   }
   

3. Consistent Cleanup Pattern:

   beforeEach(() => {
     // Create isolated timer instances for each test
     const timer = TimerController.createInstance();
   
     // Reset any static state
     MemoryTransport.resetRegistry();
   
     // Create test instances with isolated timers
     testInstance = new ClassUnderTest({
       timerController: timer
     });
   });
   
   afterEach(() => {
     // Clean up instances
     testInstance.dispose();
   
     // Reset any static state again
     MemoryTransport.resetRegistry();
   });
   

4. Time-based Assertions:

   // Before
   const startTime = Date.now();
   await someAsyncOperation();
   const elapsed = Date.now() - startTime;
   expect(elapsed).toBeGreaterThanOrEqual(expectedTime);
   
   // After
   const timer = TimerController.createInstance();
   const startTime = timer.getCurrentTime();
   
   // Start the operation without await
   const operation = someAsyncOperation();
   
   // Advance time
   timer.advanceTime(expectedTime + 10);
   
   // Now wait for the operation to complete
   await operation;
   
   // Check the operation had the expected effect
   // But don't assert based on actual elapsed time
   

By implementing these changes systematically across the test suite, we can ensure proper test isolation and eliminate the timing-related test failures.