Delta XMPP Project

Here is a simple trick you should use when starting up your Qt program. If you have read just about any of my “main.cpp” files, you may have seen it.

Ok, so say you have this:

class App : public QObject
{
        Q_OBJECT
public:
        App()
        {
                // init, do stuff, whatever
        }

signals:
        void quit();
};

int main(int argc, char **argv)
{
        QApplication a(argc, argv);
        App app;
        QObject::connect(&app, SIGNAL(quit()), &a, SLOT(quit()));
        a.exec ();
        return 0;
}

Here’s a basic application object. It can emit quit() to end the program. The constructor does the initialization, makes QWidgets, etc, and off you go.

However, there is a subtle problem. The Qt eventloop is not active during App’s constructor! The eventloop begins only once exec is called. Here are the snags of this pre-eventloop phase:

1) You can’t call any functions that require the eventloop to already exist. For example, I believe QObject::deleteLater() does nothing if the loop hasn’t started. I don’t know if you can use QTimer objects either (singleshots, however, are allowed. see below).

2) You can’t exit via QCoreApplication::quit(). This means if you decide during your initialization that you want to exit, you can’t. You need to wait for the eventloop to begin before you can do that.

We can fix this with a QTimer singleshot, an operation that fortunately does work in this phase. We simply create a start() slot in App, and set a singleshot on it from main().

class App : public QObject
{
        Q_OBJECT
public slots:
        void start()
        {
                // init, do stuff, whatever
        }

signals:
        void quit();
};

int main(int argc, char **argv)
{
        QApplication a(argc, argv);
        App app;
        QObject::connect(&app, SIGNAL(quit()), &a, SLOT(quit()));
        QTimer::singleShot(0, &app, SLOT(start()));
        a.exec ();
        return 0;
}

That’s it! Now you can do your major initialization inside start(), with the assurance that all code inside there will work as expected.

Suppose you want to do some work in another thread. Qt gives you the right tools — QThread, QMutex, QWaitCondition, and even signals and slots across threads — but there is still some tedious and error-prone work left even for basic situations.

First of all, there’s no “simple” way to know when the services of a remote thread can be referenced. QThread has a started() signal, but I find it to be nearly useless. Just because some thread has started does not mean you can start calling methods on objects in that thread. How do you know the object pointers? Have the objects even been made yet? Is there any initialization that needs to be done in the remote objects before you can start shooting signals all over the place? What we really want is a signal that is emitted not only after the thread starts, but after some of our own initialization occurs within that thread.

That said, I also consider an asynchronous initialization notification to be overkill in the general case. Starting a thread should take no time at all, nor should the initialization of some objects. Synchronizing the startup of the thread greatly decreases the complexity of the program. Similarly, I think it is worthwhile to synchronize shutdown.

Well, here we go:

class SyncThread : public QThread
{
        Q_OBJECT
public:
        QMutex m;
        QWaitCondition w;
        QEventLoop *loop;

        SyncThread(QObject *parent = 0) : QThread(parent)
        {
                loop = 0;
        }

        ~SyncThread()
        {
                stop();
        }

        void start()
        {
                QMutexLocker locker(&m);
                QThread::start();
                w.wait(&m);
        }

        void stop()
        {
                QMutexLocker locker(&m);
                if(!loop)
                        return;
                QMetaObject::invokeMethod(loop, "quit");
                w.wait(&m);
                wait();
        }

protected:
        virtual void run()
        {
                m.lock();
                loop = new QEventLoop;
                atStart();
                w.wakeOne();
                m.unlock();
                loop->exec ();
                m.lock();
                atEnd();
                delete loop;
                loop = 0;
                w.wakeOne();
                m.unlock();
        }

        virtual void atStart() = 0;
        virtual void atEnd() = 0;
};

So, what does this all mean? Well, now, instead of inheriting QThread and implementing the run() function, you can inherit SyncThread and implement the atStart() and atEnd() functions.

Suppose you make a class called MyThread that inherits SyncThread:

class MyThread : public SyncThread
{
        Q_OBJECT
public:
        SomeObject *obj;

        MyThread(QObject *parent) : SyncThread(parent)
        {
                obj = 0;
        }

protected:
        void atStart()
        {
                obj = new SomeObject;
        }

        void atEnd()
        {
                delete obj;
        }
};

Now, you can call start() to begin the thread, and SomeObject is guaranteed to be accessible when start() returns.

MyThread *thread = new MyThread;
thread->start();
thread->obj->foo();
delete thread;

Stopping the thread is also cleanly synchronous. The eventloop of the other thread is stopped, atEnd is called, and the thread itself stops, all before the delete finishes.

If you keep your atStart/atEnd reasonable, that is, you don’t put more code in them than you’d normally put into one pass of the event loop, the performance trade-off should be negligable.

Look above at MyThread again just to see how simple it is. There isn’t even a mutex! Stick to signals and slots when interacting with SomeObject, and you can have safe multithreaded code without even trying.

Ok, so we can do this now:

That’s presence using Link-Local Messaging (a.k.a. Apple’s Bonjour Chat protocol, now documented in JEP-0174). Presence is delivered via local DNS Service Discovery. Chat is done via peer-to-peer Jabber.

Doing Bonjour presence is not all that new actually. The JDNS subsystem has been able to do it since last year. What is notable this time around is that I’m doing it via the new Iris DNS API, which is an abstraction layer, and JDNS is just one possible backend. And there’s also an avatar / buddy icon. That’s good old icon_32.png from Psi.

If you want to try it yourself, have a look at the irisnet command-line tool (now committed to the CVS). Quite a few rough edges remain, but it should work. Here is what I ran:

./irisnet pserv justin@ubuntu _presence._tcp 5298
"txtvers=1,1st=Justin,port.p2pj=5298,status=dnd,
msg=coding on iris stuff,
phsh=b12eebe7b5a2f2673aba2f6eb0f3264939a78f67"
-a null:icon_32.png

Yeah, it’s pretty long. It is supposed to be a single line, but I’ve broken it up for readability. “irisnet pserv” means to publish a service. Then it goes: JID, service type, port, list of attributes. The -a means to add an extra DNS record, in this case the avatar image (yeah, iChat delivers avatars over DNS, don’t ask questions).

The value of the phsh attribute is the SHA1 hash of the avatar image in hexidecimal. I used the ‘sha1sum’ program to do this, which is present on most Linux installations at least:

$ sha1sum icon_32.png
b12eebe7b5a2f2673aba2f6eb0f3264939a78f67  icon_32.png

I don’t know when we’ll see Link-Local Messaging in Psi, but it shouldn’t be too hard to do at this point.

There are three ways Psi might locate a Jabber service. ServiceResolver covers all three, on all platforms, and makes it easy.

To demonstrate how ServiceResolver works, below you can see some output of my ‘irisnet’ commandline tool (which still in development).

The main way to find a Jabber server is via SRV. ServiceResolver looks up the SRV records of a given type on a given domain, as well as the address (A/AAAA) records for each:

$ ./irisnet rservd sapo.pt _xmpp-client._tcp
[213.13.146.7] port=5222
[213.13.146.8] port=5222

You can also lookup non-SRV (“plain”) services. This is done by pre-supplying the port to use, which is returned with each address record found:

$ ./irisnet rservp gmail.com 5222
[216.239.57.107] port=5222
[64.233.161.107] port=5222
[64.233.171.107] port=5222

Lastly, suppose there’s a JEP-0174 / iChat service instance on the LAN called someguy@macbox. In this case, DNS-SD is used:

$ ./irisnet rservi someguy@macbox _presence._tcp
[192.168.1.156] port=5298

There may be three different input methods, but there is only one output method: a list of address & port pairs. The list is always sorted in the order in which the addresses should be tried. Priorities and weights are taken care of for you.

With cross-platform capability, pluggable backends, and seamless Bonjour support, the Iris DNS system is shaping up to be one of the most simple and capable such systems in existence. I suppose I don’t feel so bad about it taking forever to finish, because we really have something cool here.

#ifdef QT_GUI_LIB
# include <QApplication>
#endif

inline bool is_gui_app()
{
#ifdef QT_GUI_LIB
        return (QApplication::type() != QApplication::Tty);
#else
        return false;
#endif
}

This is enough if you don’t plan to actually touch the Qt GUI API. Otherwise, you’ll have to drag that QT_GUI_LIB ifdef all over your code.

(I’m going to use this to determine whether or not I should set up the Windows console control handler.)

Alright, after a lot of research, I’ve concluded that cross-platform process termination is a tricky beast. It is not possible to have a clean and balanced solution for all platforms, but at the end of this post you’ll find a decent guideline.

On Windows, at the lowest level, a process can only be terminated abruptly, with no chance for the process to cleanup. In other words, it is like Unix’s kill -9. To have a clean termination, the process has to be hinted somehow, and this differs depending on the situation. A GUI app can have a WM_CLOSE event sent to its window. A console app cannot receive WM_CLOSE, so Windows has a separate facility (via SetConsoleCtrlHandler) that the app can use for determining if the parent console window is being closed, or if Ctrl-C has been pressed. What’s interesting about these events, particularly WM_CLOSE, is that they don’t explicitly mean a process should end. For example, Psi, with the system tray active, will never quit from a WM_CLOSE. If the app doesn’t have a GUI window or a console, then all you can do is terminate the process. This is why the Windows Task Manager has two listings from which to end tasks/processes. One sends WM_CLOSE (and, in the case of a console, the proper notification is relayed to the app running inside it), the other outright kills the process.

On Unix (which includes Mac OS X), there is SIGTERM, and it works for any process, console or GUI. The SIGTERM signal is a request to terminate, giving the app the opportunity to perform last minute tasks.

Now, in addition to process termination, there is the notion of “window system shutdown”, which is supported on Windows, X11, and Mac. This is a very important topic, because, at least on Windows and X11, handling of the window system shutdown is generally required for a clean exit (I say “generally”, because on X11 you have a second chance, I’ll explain next). As I already discussed, there is no mechanism to be friendly terminated on Windows. And so it should not be a surprise that Windows uses another hint here, WM_ENDSESSION, to tell you that the system is shutting down. In fact, it first uses a different hint, WM_QUERYENDSESSION, to give your app a chance to perform user interaction (e.g. “Save this Document?”) and possibly reject the shutdown.

All three window systems have varying features. X11 is the most advanced here, and actually has two phases, plus a mechanism for storing session information (like window sizes/positions). I suspect Windows and Mac don’t have advanced session management because they are not remote and can therefore just suspend/hibernate (not to mention that suspending an X server is a rather recent thing). The reason you must handle these shutdown events is because Windows has no other way of terminating your app in a friendly way, and with X11 your app probably won’t get a SIGTERM when the server shuts down. On Mac, I don’t know what happens. Therefore, because I don’t have another answer for you, you must handle the shutdown event on Mac.

Now, you might be wondering about why you don’t get a SIGTERM on unix when X11 goes down. Well, think about it, the X server (graphic display) and X client (your app) don’t need to be in the same place. Losing the X server simply means losing a socket. This doesn’t rule out the possibility of some smart window manager sending SIGTERM to local apps, but you can’t rely on this. If you handle nothing else, handle this “X display is gone” scenario, because it should cover all shutdown scenarios. If you lose a remote X connection, I assure you that you won’t get to do the nice X11 window system shutdown procedure (including the session management goodness, sorry). This scenario is what I was referring to about the “second chance”.

What does this all come down to?

- If you are a GUI app, handle window system shutdown (including “display is gone”).

- If you are on Unix, handle signals. You should do this even if you are a GUI app, simply because you can.

- If you are a Windows console app, handle the Windows console events, which incidentally also include window system shutdown events.

- If you are a Windows GUI app, don’t handle console events, that’s just confusing.

I just threw together some code for handling when a process is requested to terminate. The motive is so I can quit a console application cleanly. With GUI apps, detecting when the user wishes to exit is a trivial task, since any GUI toolkit worth anything will tell you when a window closes (via an ‘X’ button or ALT-F4 or whatever). However, console apps, especially those that run indefinitely, and barring those that have interactive UIs such as curses or prompts, are commonly exited by using Ctrl-C. Fortunately, tracking this kind of event wasn’t too bad to implement. Windows has a handler function specifically for console breaking, and Unix has signals (I trap SIGINT, SIGHUP, and SIGTERM, per the glibc manual).

The JDNS commandline tool has an argument to specify how long it should run, which is just plain silly. And my network interface monitoring test program had a similar option too… Until tonight! Now I can just hit ctrl-C and the app will quit. Clean exits are essential when valgrinding, and so this will make my life much easier.

And now that I’ve thought about it some more, this code may also be useful in GUI apps like Psi, for when a system shutdown occurs. However, I think the Windows code I have is for console only, so that will have to be resolved before it is useful there..

Creating C++ classes in global space is generally a bad idea, because the order of construction is undefined. On top of that, constructors require execution, thus slowing down your program startup time. Enter Q_GLOBAL_STATIC. It works like this:

Q_GLOBAL_STATIC(Foo, get_foo)

Write this in your global space, and it will define a function called get_foo() that essentially does “new Foo” (if it has not done so already) and returns it. Now, instead of using the object directly, you use the function. This puts you in control of when your global objects are created, and gives you lazy-loading to boot. An atomic integer operation is used for thread-safe construction. In addition, a static template class “wrapper” is defined for your type, so that your object will be properly destructed on exit, just like a normal global type. Granted, this is a slight bit of overhead, but probably less than any object you’d use it with. There’s even Q_GLOBAL_STATIC_WITH_ARGS().

You probably shouldn’t be using globals anyway, but sometimes you need them. Q_GLOBAL_STATIC seems particularly useful with QMutex, the bare-minimum to get you off the ground for creating thread-safe singletons.

Right now I’m working on detecting when network interfaces come and go, so that, for example, Psi can go online when your network connection becomes active. There is a nice debate as to what constitutes a network connection, but I think for our purposes I’ve decided this will be a non-loopback interface bound to a gateway. This code will be part of IrisNet, which is the new network subsystem for Iris.

Time for platform-specific coding fun again.

You can’t delete a QObject directly unless you do it from the thread it lives in. However, you can use object.moveToThread(0) to disassociate the object from any thread, allowing it to be deleted from anywhere. The drawback is that you can’t use queued signals/slots, but for the kind of object you would do this with it is probably not an issue.

QCA uses this trick to disassociate backend provider QObjects from threads so that they can be cleaned up from any thread. This is mainly for the public key front-end objects which are not QObjects, and so the user has an expectation that they can be copied between threads.