Go to the first, previous, next, last section, table of contents.
"Longer, lower, wider, faster than anything else in its price class!"
-- Excerpt from early 1970s car advertisement.
In a world with literally thousands of software platforms, what makes Muq
special? When is Muq an appropriate design choice?
Muq goes several design generations beyond (say) MOO or Java in providing support
for applications characterized by these five requirements:
Multi-user: The application involves close interaction between many users.
Persistent state: The application runs for years rather than hours.
Complex data: The application dataset doesn't neatly reduce to tables.
Complex code: The application requires full-strength programming
languages and per-user customizibility.
Distributed: The application requires coordination of state, code and
users distributed over many network-linked machines.
If your application fits this profile, picking Muq as your platform
may reduce your development time by an order of magnitude or more
while also improving the reliability and robustness of your
Traditional software platforms such as C or Java provide very little
support for any of the above requirements, but the combination is
becoming increasingly common in the Internet age.
To support these five requirements well, especially in conjunction,
a software platform must really have been designed with them in
mind from the very beginning: The requirements affect thousands
of design decisions throughout the implementation.
Muq was so designed, building on many years of experience using,
maintaining, administering and writing online communities and
Here are some of the specific facilities Muq provides which support
the target application space:
Binary disk-based persistent state: All state information is
automatically persistent, automatically backed up regularly,
and data which haven't been used recently are automatically
swapped to disk to conserve physical ram. Up to forty percent
of the code for many applications is devoted to implementing
this functionality: In the Muq platform, that part is done
and debugged before you start.
Well-factored state: The state information is divided up
logically among different host files. For example, each standard Muq
system library has its own file, as does each user. This makes it
easy to install updated system libraries, easy to roll back a user's
state without rolling back the entire database, and easy to move
a user from one server to another. Competing platforms usually
roll all the state into one unmanageable blob: In a production
environment, this becomes a nightmare the first time a user
complains, "I accidentally deleted all my stuff, can you
restore it from backup for me?".
Multiple application languages: Experience has shown that there
is no such thing as a one-size-fits-all programming language:
Different users need different languages, and a large multi-user
application has to recognize this. Most competing platforms
force all users and programmers to use a single syntax: Muq
is specifically designed to support multiple programming syntaxes,
and in fact allows unprivileged users to implement new programming
syntaxes without compromising system reliability or security,
thanks to a core backend generator which guarantees to
produce only valid compiledFunction objects.
Full-featured application languages: Competing platforms often have
application languages added as an afterthought, grown without design
from some primitive macro language. You are often stuck trying to
build real applications in a toy language. Muq's application
languages were designed to be fully modern and full featured from
the very beginning, and include a full complement of modern
data and control structures and facilities, up to and including
object oriented programming with multiple inheritance.
Automated storage management ("garbage collection"): Discarded
datastructures are automatically detected and recycled. Typical
C/C++ problems of memory leaks, hanging pointers and clobbered
memory are inherently impossible. This alone can cut your
debugging time in half on many projects.
Multi-user design: Every datum is owned by some specific user,
can by default only be modified by that user, and can be hidden
from other users. Every operation is designed to respect these
(and many additional) security concerns. Competing platforms
typically lack such designed-in security walls, and retrofitting
them at the application code level can be problematic at best.
Transparent distributed operation support: Muq's muqnet support
merges selected Muq server processes into a single logical computer.
The processes can be on the same machine or on opposite sides of the
Internet. All threads can communicate with each other in exactly the
same way regardless of whether they are local or remote. All threads
can access all data in exactly the same way regardless of whether they are
local or remote. (Naturally, security safeguards apply: Private data
is protected from access by other users, whether it is local or remote.)
Painstaking attention to security and privacy concerns throughout
the design and implementation. For example:
This support can by itself cut the implementation time for a distributed
application in half or more.
Users are automatically and transparently assigned public/private keypairs.
Private keys are stored in a special datatype unreadable even to the user
Muqnet traffic is automatically and transparently 256-bit blowfish encrypted.
Muq tells the recipient of each inter-user call who the sender was. This
information is validated by internal server logic for intra-server calls
and by public key techniques for inter-server calls, reducing spoofing
problems by a large factor.
Migration support: In a distributed application with perhaps thousands
of servers spread across the internet and state persisting for years,
what happens when a server must move from one IP address to another?
Muqnet logic handles this completely transparently and automatically.
Scalability: Muq and Muqnet are designed to be as scalable and
reliable as realistically possible. In particular, muqnet needs no
central node, which might constitute a potential single point of
failure capable of bringing the entire system down. Communication is
peer-to-peer with no inherently distinguished server nodes. (There
are of course be some muqnet nodes pragmatically known to be stable
and secure, and others which are online from dynamic IP addresses on
laptops or public workstations, and they are in consequence actually
Robustness: For example, the underlying Muq C-coded server is designed by
the "mechanism not policy" rule: Customization can typically be done
without touching the C code, which vastly reduces the probability of
introducing nasty crashing bugs during development. Competing platforms
typically put most policy decisions in the C server code, making constant
tweaking of it a requirement and constant crashes a fact of life.
Careful attention to space efficiency issues throughout. For example, Java
requires over 100 heap bytes to store the string "bloated"; Muq takes
no heap bytes at all -- it is stored as an immediate value within a Muq
64-bit variable. The state of multi-user applications is dominated
by short strings, so this alone can reduce your application space
consumption by a factor of ten. Since Muq swaps
unused objects to disk while Java keeps them in memory, the practical
ram consumption ratio may run as high as one hundred.
Similar attention to time efficiency issues. For example, Muq has
less than 1/3 the instruction dispatch overhead of most tinyMU*
Portable: Muq produces exactly the same results on all machines.
(Java, for example, produces different results on 32-bit vs 64-bit
machines.) Muq db files are designed to be portable between all
supported machines without any explicit conversion needed. (For
speed, the db is internally big-endian on big-endian machines and
little-endian on little-endian machines: This allows the dbfiles
to be memory-mapped and accessed at high speed. But no explicit
user-invoked conversion is used.)
Muq runs 64-bit throughout, on all machines: This in an
investment in the future, since 32-bit era is clearly coming to an
end. Running 64-bit on all platforms (including x86) costs about a
15% slowdown on compute-intensive tasks, but means that your
applications are guaranteed to port to next-generation 64-bit machines
completely painlessly when the time comes. (Dbs developed on 32-bit
platforms such as x86 Java may be nightmares to port to 64-bit
platforms.) You can also run Muq dbs indifferently on today's Intel,
Sparc and MIPS platforms without porting issues.
Muq addresses such issues as what happens to instances of a
class when it is redefined. This is a crucial issue in a
production multi-user application with state intended to
persist for years, but is completely ignored by platforms
such as Java or C++, which expect all objects to vanish
at the end of a program run of a few hours. Coding around
this problem on a platform not designed to deal with it may
easily add months to your application development.
Muq is a new platform designed for the needs of a new millenium, with
sufficient design hooks in place to keep it at the leading edge for
years to come. Muq represents the leading edge of the future:
Competing platforms represent the receding edge of the past.
Go to the first, previous, next, last section, table of contents.