Distributed Computing

Est. read time: 2 minutes | Last updated: January 17, 2025 by John Gentile

Contents

Virtualization and Containers
Cloud
- Amazon Web Services (AWS)
High-Performance Network Programming
- Packet FEC in lieu of Retransmission
References

Architectures and design patterns for systems that are data intensive, as well as distributed computing systems. For a great overview, and impetus for AWS, see Amazon’s Distributed Computing Manifesto. amazon distributed computing Manifesto diagram

Virtualization and Containers

Virtual Machines (VM)

Containers

Useful Docker Commands

Restart last container created: docker start $(docker ps -ql)
- Reattach terminal & stdin to last: docker attach $(docker ps -ql)
List currently running Docker instances: docker ps -a
Stop all containers: docker kill $(docker ps -aq)
Delete all containers, including its volumes they use: docker rm -vf $(docker ps -aq)
Delete all docker images: docker rmi -f $(docker images -aq)
- NOTE: you should remove all containers before removing all the images from which the containers were created.
Delete everything (removing all unused containers, volumes, networks, images, etc.) and reclaim disk space (essentially remove everything!), use: docker system prune -a --volumes

References

Cloud

Amazon Web Services (AWS)

High-Performance Network Programming

Packet FEC in lieu of Retransmission

When latency is key (can’t wait/block for packet loss) in lossy networks (e.x. WAN, intermittent links, etc.), Forward Error Correction (FEC) techniques (similar to those used at the physical layer) can be applied at the network layer. For instance in SD-WAN FEC, lost packets can be recovered on a link by sending extra “parity” packets for every $N$ packets. See more details on Information Theory.