Links tagged “distributed”

pythonhosted.org

Documentation for osBrain, a general-purpose multi-agent system module in Python whose agents run as separate processes and exchange messages over ØMQ and Pyro4.

igvita.com

Builds a blockchain from first principles, showing how digital signatures, proof of work and the longest-chain rule settle transactions without a trusted third party.

blog.gopheracademy.com

An introduction to Glow, a pure-Go map-reduce framework for standalone and distributed data processing.

ceph.com

Open-source distributed storage system offering object, block and file interfaces from a single cluster built on commodity hardware.

gluster.org

Free and open-source scalable network filesystem that pools commodity storage into large distributed volumes for media streaming and data analysis.

kafka.apache.org

The official Apache Kafka documentation, covering the distributed commit log's design, configuration, operations and client APIs.

davidmarquis.wordpress.com

A design for reliable, exactly-once delivery to multiple consumers built on Redis lists and sets rather than its fire-and-forget pub/sub.

consul.io

Service networking: discovery, health checking and secure connectivity between services across data centres.

mesos.apache.org

Project page for Apache Mesos, a cluster manager abstracting CPU, memory and storage away from machines to run distributed systems; now retired to the Apache Attic.

blueflood.io

Project site for Blueflood, an open-source distributed metrics store built on Cassandra at Rackspace, with near-real-time ingestion, querying and background rollups.

couchdb.apache.org

Project page for Apache CouchDB, a document database with an HTTP/JSON API and a replication protocol that syncs data across servers, browsers and mobile devices.

couchbase.com

Home page of Couchbase, a distributed NoSQL document database now positioned as an operational data platform with vector search and real-time analytics.

spark.apache.org

A multi-language engine for data engineering, data science and machine learning, on one machine or a cluster.

github.com

Python distributed queue for huge numbers of one-off tasks, built on gevent and ZeroMQ with resumable iterators so the master never buffers the whole job list.

christophermeiklejohn.com

A curated bibliography of distributed systems papers grouped by topic — consensus, consistency, data structures, programming models — with a short list of recommended books.

elasticsearch.org

Elasticsearch is a powerful open source search and analytics engine that makes data easy to explore.

slideshare.net

Slide deck cataloguing distributed-systems patterns: scalability and consistency trade-offs, fail-over and replication, then caching, sharding and NoSQL for state.

tsung.erlang-projects.org

Home page of Tsung, an open-source distributed load-testing tool written in Erlang that stresses HTTP, WebDAV, SOAP, database, LDAP and XMPP servers.