Week 14 Update: Scheduler Testing Complete & Experimentation Begins

Overview

This week, we wrapped up the testing phase for our schedulers, ensuring that all scheduling paradigms functioned as expected. After fixing a few final bugs and making minor improvements, we shifted our focus to running experiments comparing the performance of the three schedulers: First-Come, First-Served (FCFS), Random, and our custom workload-balancing scheduler.

Finalizing Scheduler Tests

• Test Completion:
  • We finished writing the last batch of Google Test cases to validate that each scheduler handles edge cases and failure scenarios correctly. These tests focused on ensuring:
    • Correct host selection and invocation scheduling.
    • Proper error handling for resource constraints (e.g., when all hosts are full).
    • Accurate image management.
• Bug Fixes:
  • Minor issues were discovered during final test runs, such as a bug in our Random scheduler where some hosts were never selected due to an off-by-one error.
  • We also fixed a small performance issue in the workload-balancing scheduler related to how host load was calculated during fast invocations.
• Test Pass Rate:
  • All tests now pass consistently under normal and edge-case conditions, with a particularly strong emphasis on thread safety and state consistency.

Starting Scheduler Experiments

With the schedulers fully validated, we transitioned to running experiments to compare their performance:

• Experiment Setup:
  • We prepared a set of synthetic workloads designed to test the schedulers under different conditions:
    • Low load with balanced host distribution.
    • High load with varying invocation sizes and rates.
    • Resource contention scenarios (e.g., limited CPU or memory).
• Initial Results Collection:
  • Early results are being logged for each scheduler across different metrics, such as:
    • Average invocation completion time.
    • Resource utilization (CPU, memory, disk).
    • Host load distribution.
• Goal of the Experiments:
  • Our aim is to identify which scheduler performs best under specific conditions (e.g., fairness vs. efficiency trade-offs).
  • The results will inform future improvements to the schedulers and help fine-tune configurations for real-world usage.

Next Steps

• Further Experimentation:
  • Run additional simulations with more diverse and larger workloads to capture a wider range of performance data.
  • Begin analyzing the logs to draw conclusions about the trade-offs between the schedulers.
• Performance Tuning:
  • Depending on initial results, consider optimizing certain schedulers to improve efficiency, especially in resource-limited environments.
• Prepare Documentation:
  • Start preparing detailed reports of the experiment results and scheduler performance to share with stakeholders and users.

---

This week marked the transition from stabilization to evaluation. By completing scheduler validation and starting the first round of performance experiments, we’re now in a great position to understand how our scheduling strategies scale under different workloads and refine them accordingly.