Archive for Programming

Parallel builds with qmake

So you’d like to run make -j4 on your qmake project (or make -j3, make -j2, hi google). Yes, it is possible, but like many things qmake, it is not documented. As usual, the best way to learn about qmake is to see how Qt itself uses it. Actually, I recommend Qt’s own build files as a primary source for qmake information anyway, as if you know where something ought to be in a project then it is faster to find it than going through the formal manual.

In a SUBDIRS project, you may have something like this:

SUBDIRS += library unittest examples

Change it to this:

# depend on variables, not actual directories
SUBDIRS += sub_library sub_unittest sub_examples

# specify the directories and dependencies
sub_library.subdir = library
sub_unittest.subdir = unittest
sub_unittest.depends = sub_library
sub_examples.subdir = examples
sub_examples.depends = sub_library

Have fun.

Comments (2)

Analysis of writeDatagram

Here is the source code for Qt 4.2.2′s QUdpSocket::writeDatagram:

qint64 QUdpSocket::writeDatagram(const char *data, qint64 size,
    const QHostAddress &address,
    quint16 port)
{
    Q_D(QUdpSocket);
#if defined QUDPSOCKET_DEBUG
    qDebug("QUdpSocket::writeDatagram(%p, %llu, \"%s\", %i)", data, size,
           address.toString().toLatin1().constData(), port);
#endif
    QT_ENSURE_INITIALIZED(-1);
    qint64 sent = d->socketEngine->writeDatagram(data, size, address, port);
    if (sent >= 0) {
        emit bytesWritten(sent);
    } else {
        d->socketError = d->socketEngine->error();
        setErrorString(d->socketEngine->errorString());
        emit error(d->socketError);
    }
    return sent;
}

If this code looks sane to you, then you’ve already forgotten your lesson on Delayed Signals. Here we have a function that should clearly be using delayed signals, since the signals are being used “as a means of returning or indicating a result of a requested operation” (I need a shorter term for this :) ). The linked article already goes over why delayed signals should be used with request-based operations, but I’ll go over the concept again below in an example with QUdpSocket.

Suppose you wanted to put “Hello World!” inside of a UDP packet. Suppose you wanted to send such a packet to some computer on port 2000. Suppose you wanted to send a lot of these. I suppose you’d end up with something like PacketSender:

class PacketSender
{
        Q_OBJECT
public:
        PacketSender()
        {
                sock = new QUdpSocket(this);
                connect(sock, SIGNAL(bytesWritten(qint64)),
                        SLOT(sock_bytesWritten(qint64)));
                connect(sock, SIGNAL(error(QAbstractSocket::SocketError)),
                        SLOT(sock_error(QAbstractSocket::SocketError)));
        }

        void start(const QList &addrs)
        {
                QByteArray packet = "Hello World!";
                total = addrs.count();
                foreach(QHostAddress addr, addrs)
                {
                        sock->writeDatagram(packet.data(), packet.size(),
                                addr, 2000);
                }
        }

signals:
        void finished();
        void error();

private slots:
        void sock_bytesWritten(qint64)
        {
                --total;
                if(total == 0)
                        emit finished();
        }

        void sock_error(QAbstractSocket::SocketError)
        {
                delete sock;
                emit error();
        }

private:
        QUdpSocket *sock;
        int total;
};

With PacketSender, you instantiate the object, connect your signals, and call start() with a list of addresses to send the packets to. PacketSender will emit finished() when all packets have been sent successfully, or error() if it runs into any problems while sending. Either signal shall indicate the end of the operation. Example use:

PacketSender *sender = new PacketSender;
connect(sender, SIGNAL(finished()), ...);
connect(sender, SIGNAL(error()), ...);
QList addrs;
addrs += "192.168.0.1";
addrs += "192.168.0.2";
sender->start(addrs);

The code for PacketSender looks good. In fact, this code is probably perfect for its intent. However, there are some problems here, and it is because we inherit these problems from QUdpSocket. This is quite common among my Qt guidelines. Misbehaving classes must be compensated for if you use them inside of your own classes, otherwise users will have to compensate for *your* class. Ultimately, either the developer or the user of a class will have to make things work. My philosophy is to put the greater burden on the developer of the class, as a class is developed once but used many times.

The first problem with PacketSender is that it does not obey the delayed signals guideline. It looks like it should, but QUdpSocket’s behavior causes PacketSender to break the rules. For example, finished() might be emitted before start() returns.

One simple solution would be to delay the call to writeDatagram, by reworking it like this:

...
public:
...
        void start(const QList &addrs)
        {
                addrs = _addrs;
                QTimer::singleShot(0, this, SLOT(real_start()));
        }
...

public slots:
...
        void real_start()
        {
                QByteArray packet = "Hello World!";
                total = addrs.count();
                foreach(QHostAddress addr, addrs)
                {
                        sock->writeDatagram(packet.data(), packet.size(),
                                addr, 2000);
                }
        }
...
private:
...
        QList addrs;
...

The second problem with PacketSender is that it does not obey the signal safety guideline, and this stems from its assumption that QUdpSocket is signal-safe. This assumption should never be made unless the class or a specific function/signal is known or labeled to be signal-safe (no Qt classes are). Funny enough, by peeking into the code, we can see that writeDatagram()’s code is actually signal-safe. Also, judging by the fact that the function is guaranteed to emit either bytesWritten() or error(), we can probably assume that these signals are never emitted as a result of any other operation.

However, PacketSender::start() is not signal-safe due to QUdpSocket not practicing delayed signals. If PacketSender is deleted during the finished() or error() signal, start() will still be on the call stack, and the next iteration of the foreach loop will cause a crash. By employing the delayed signals fix, PacketSender should in theory be signal-safe, because then start() won’t be on the call stack.

That said, since QUdpSocket is not documented as being signal-safe, I don’t think it is something we should rely on. Here’s the fix:

...
public:
        ~PacketSender()
        {
                sock->disconnect(this);
                sock->setParent(0); // prevent auto-delete
                sock->deleteLater();
        }
...

Yay, PacketSender is now signal-safe and delayed-signal compliant! There should be a seal of approval for this kind of thing.

[Side note: there is another problem not related to our discussion, but I must point it out: it is quite possible that QUdpSocket can only accept a finite number of writes at a given time. The signal-based reply leads us to believe that it would probably queue all packets internally and send them as kernel write space becomes available, but a peek at the code shows that writeDatagram() will report errors immediately (synchronously). This likely means it is not queuing at all, and attempting to call PacketSender::start() with a large list of addresses will likely fail.]

Amusingly, writeDatagram() also has a return value, which essentially indicates the same thing as the signal. By looking at the code, we can see that we only need to use either the signals or the return value. However, by looking at only the documentation, it would appear that we may need to check both values (and we probably still should, since we have no guarantee that the writeDatagram() implementation will always work this way). What happens if bytesWritten() is emitted but writeDatagram() returns an error? Talk about confusing…

Really, Trolltech ought to either get rid of the signals or the return value. Getting rid of the signals is the easier solution, but is less abstract as it relies on the underlying socket implementation being able to know the result of the write operation immediately. Getting rid of the return value is a more abstract solution, but something really needs to be done about those non-delayed signals.

My policy when interacting with Qt classes lately is to compensate for these kinds of problems at the earliest level (usually within whichever class of mine is directly using the Qt class), so that code stays clean all the way up. The IrisNet TcpStream and UdpStream classes, for example, internally assume the worst from QTcpSocket and QUdpSocket and supply proper workarounds.

Comments (2)

Synchronized Threads (Part 3)

The first two parts in this series dealt with synchronously starting and stopping threads. Now we’ll bring in what we know about QMetaObject::invokeMethod and dynamic return types to perform synchronous function calls across threads.

Synchronous function calls are not for everyone. One thread (the caller) waits while another thread performs and completes a function call. If you are using threads for performance reasons, making use of synchronous calls between your threads will probably defeat the point of having threads at all.

So who are they for, then? Synchronized calls can come in handy when you’re using threads for reasons other than performance. There are situations where you simply must have a thread to perform an operation, because the operating system or some library requires it. Other times, maybe you want to perform a task asynchronously but only a “blocking” API is offered. In these cases, a thread is your ticket, and it has nothing to do with performance. Additionally, if you have infrequent communication between threads, then a synchronous call might not have any performance impact anyway.

Let’s now upgrade SyncThreadAgent:

class SyncThreadAgent : public QObject
{
        Q_OBJECT
public:
        SyncThreadAgent(QObject *parent = 0) : QObject(parent)
        {
                QMetaObject::invokeMethod(this, "started",
                        Qt::QueuedConnection);
        }

signals:
        void started();
        void call_ret(bool success, const QVariant &ret);

public slots:
        void call_do(QObject *obj, const QByteArray &method,
                const QVariantList )
        {
                QVariant ret;
                bool ok = invokeMethodWithVariants(obj, method, args,
                        &ret, Qt::DirectConnection);
                emit call_ret(ok, ret);
        }
};

The call_do() slot calls a method for us. The call_ret() signal is emitted when the call is completed. This is not asynchronous, as the method is called with DirectConnection. However, since SyncThreadAgent exists in the worker thread, which could be busy at any given moment, it is intended that call_do() will be invoked using QueuedConnection. This means that the process will end up being asynchronous, from the main thread’s point of view. The request to call will be queued until the worker thread returns to the event loop, the call will then be executed, and then the signal for completion will be emitted back.

We can then convert this process into a synchronous call, by simply putting the main thread to sleep until call_ret() is emitted from the worker thread.

Below is a reworked version of SyncThread using the new SyncThreadAgent features. It is directly copied from QCA, so it is a bit more advanced due to the Private class. Notable changes from the previous article are highlighted.

class SyncThread : public QThread
{
        Q_OBJECT
public:
        SyncThread(QObject *parent = 0);
        ~SyncThread();

        void start();
        void stop();
        QVariant call(QObject *obj, const QByteArray &method,
                const QVariantList &args = QVariantList(), bool *ok = 0);

protected:
        virtual void atStart() = 0;
        virtual void atEnd() = 0;

        // reimplemented
        virtual void run();

private:
        class Private;
        friend class Private;
        Private *d;
};

// implementation

class SyncThread::Private : public QObject
{
        Q_OBJECT
public:
        SyncThread *q;
        QMutex m;
        QWaitCondition w;
        QEventLoop *loop;
        SyncThreadAgent *agent;
        bool last_success;
        QVariant last_ret;

        Private(SyncThread *_q) : QObject(_q), q(_q)
        {
                loop = 0;
                agent = 0;
        }

private slots:
        void agent_started();
        void agent_call_ret(bool success, const QVariant &ret);
};

SyncThread::SyncThread(QObject *parent)
:QThread(parent)
{
        d = new Private(this);
        qRegisterMetaType("QVariant");
        qRegisterMetaType("QVariantList");
}

SyncThread::~SyncThread()
{
        stop();
        delete d;
}

void SyncThread::start()
{
        QMutexLocker locker(&d->m);
        Q_ASSERT(!d->loop);
        QThread::start();
        d->w.wait(&d->m);
}

void SyncThread::stop()
{
        QMutexLocker locker(&d->m);
        if(!d->loop)
                return;
        QMetaObject::invokeMethod(d->loop, "quit");
        d->w.wait(&d->m);
        wait();
}

QVariant SyncThread::call(QObject *obj, const QByteArray &method,
        const QVariantList &args, bool *ok)
{
        QMutexLocker locker(&d->m);
        bool ret;
        ret = QMetaObject::invokeMethod(d->agent, "call_do",
                Qt::QueuedConnection, Q_ARG(QObject*, obj),
                Q_ARG(QByteArray, QByteArray(method)),
                Q_ARG(QVariantList, args));
        Q_ASSERT(ret);
        d->w.wait(&d->m);
        if(ok)
                *ok = d->last_success;
        QVariant v = d->last_ret;
        d->last_ret = QVariant();
        return v;
}

void SyncThread::run()
{
        d->m.lock();
        d->loop = new QEventLoop;
        d->agent = new SyncThreadAgent;
        connect(d->agent, SIGNAL(started()), d, SLOT(agent_started()),
                Qt::DirectConnection);
        connect(d->agent, SIGNAL(call_ret(bool, const QVariant &)), d,
                SLOT(agent_call_ret(bool, const QVariant &)),
                Qt::DirectConnection);
        d->loop->exec ();
        d->m.lock();
        atEnd();
        delete d->agent;
        delete d->loop;
        d->agent = 0;
        d->loop = 0;
        d->w.wakeOne();
        d->m.unlock();
}

void SyncThread::Private::agent_started()
{
        q->atStart();
        w.wakeOne();
        m.unlock();
}

void SyncThread::Private::agent_call_ret(bool success, const QVariant &ret)
{
        QMutexLocker locker(&m);
        last_success = success;
        last_ret = ret;
        w.wakeOne();
}

With SyncThread, you can start, stop, and call a method in another thread while the main thread sleeps. The only requirement is that the methods be declared as slots.

Below is a contrived example, where we have an object in another thread that increments a counter over a some interval, using the Qt event loop, and provides a method to inspect the value.

First, the Counter object:

class Counter : public QObject
{
        Q_OBJECT
private:
        int x;
        QTimer timer;

public:
        Counter() : timer(this)
        {
                x = 0;
                connect(&timer, SIGNAL(timeout()), SLOT(t_timeout()));
        }

public slots:
        void start(int seconds)
        {
                timer.setInterval(seconds * 1000);
                timer.start();
        }

        int value() const
        {
                return x;
        }

private slots:
        void t_timeout()
        {
                ++x;
        }
};

Looks like a typical object, no surprises.

Now to wrap Counter with SyncThread. We went over how to do this in the first article, and it is very straightforward:

class CounterThread : public SyncThread
{
        Q_OBJECT
public:
        Counter *counter;

        CounterThread(QObject *parent) : SyncThread(parent)
        {
                counter = 0;
        }

        ~CounterThread()
        {
                // SyncThread will stop the thread on destruct, but since our
                //   atStop() function makes references to CounterThread's
                //   members, we need to shutdown here, before CounterThread
                //   destructs.
                stop();
        }

protected:
        virtual void atStart()
        {
                counter = new Counter;
        }

        virtual void atStop()
        {
                delete counter;
        }
};

We can then use it like this:

CounterThread *thread = new CounterThread;

// after this call, the thread is started and the Counter is ready
thread->start();

// let's start the counter with a 1 second interval
thread->call(thread->counter, "start", QVariantList() << 1);
...

// after some time passes, let's check on the value
int x = thread->call(thread->counter, "value").toInt();

// we're done with this thing
delete thread;

Do you see a mutex anywhere? I didn’t think so.

Comments (2)

Signal Retraction

Qt 4 introduced easier ways to queue method calls until the next eventloop cycle. Now you can connect signals and slots together with QueuedConnection. You can use the connectionless QMetaObject::invokeMethod() with the QueuedConnection mode as well. In the old days, a delayed method call took more work. Usually you would store argument values somewhere, use a QTimer to invoke a special slot at a later time, and this slot would then take the original argument values and call the method you wanted to call in the first place. With Qt 4, you can get this stuff in one line.

Anyway, delayed calls are often a good thing. They are most useful for assisting with delayed signals (which can also aid in making your object signal-safe). In fact, if you wanted to go overboard, you could emit all of your signals late. This would make your object pretty darn safe, just not very optimized.

There is a price to all of this delayed business though. One major issue is that the state of the object may have changed between the emit and the time of receipt. Maybe a certain object expects you to inspect one of its properties from within a slot connected to one of its signals. If you were to make this a QueuedConnection, it might not be possible to use this object correctly. Of course, given that the connection mode is the user’s choice, hopefully no new objects are being written that rely on DirectConnection. If they are, they’d better be documented as such.

Unfortunately, there are some cases where getting the state right just isn’t possible, at least not in a clean way. This is when you have object re-use or object deletion.

Object Re-use

Take, for example, QTcpSocket. Imagine you set up your signals using QueuedConnection, then call connectToHost(“server1″…), abort(), connectToHost(“server2″…) (in that order), and server1 is up but server2 is down. There is a race condition, whereby connected() might be emitted (for server1), and then error() (for server2), but your application receives both of these signals after calling the second connectToHost(). Since you cannot associate these signals with a particular “usage” of QTcpSocket, the assumption is that you were able to connect to server2. In fact, if it takes some time before the connection to server2 results in an error, your application might be confused for awhile.

One solution to this problem might have been to have connectToHost return a context id, which would then be present in any subsequent signals. This is the natural approach to an asynchronous system. Of course, this is also ugly and would defeat much of the clean fun of Qt programming, so we won’t give this any further consideration. :)

Anyhow, this problem is not limited to making QueuedConnection signals with QTcpSocket. It is conceivable that QTcpSocket might use some queued stuff internally. Indeed, I’ve read the code and I’ve seen QMetaObject::invokeMethod() in there. This has the same kind of potential, where actions might be queued, and they are still in the queue even if you reset the object for a new use.

The answer? Don’t use QueuedConnection on a stateful object (like QTcpSocket), and delete the object if you want to cleanly re-use it. Any queued business will die with the object. Yes, this means abort() is useless to you.

Object Deletion

Deleting an object will stop any internal queued stuff.

However (and this is a big one), “externally” queued stuff, such as a signal/slot connection that you made using QueuedConnection, will not be stopped. Qt only verifies the signal and slot connection during the time of emit. When it is time for the slot to be called, only the receiver needs to still exist. This means that if you are using QueuedConnection, it is entirely possible that you may delete an object, only to later have one of your slots be called because the sender emitted a queued signal just before you deleted it. Fortunately, Qt does zero out the sender() if it no longer exists at the time of the slot.

The answer? Don’t use QueuedConnection on a stateful object if you want to be able to delete it without it haunting you.

Signal Retraction

Sometimes, you need to queue an action internally. We already went over the benefits. The problem is that by doing queued things, your object is not easily trusted for re-use. If someone wants to re-use your object, then they are better off deleting it and starting fresh. Can we improve the situation? Yes! With Signal Retraction.

Signal retraction is where a queued outbound signal is canceled when the object is re-used. For example, if QTcpSocket practiced signal retraction, then calling abort() would cancel all queued activities, such that we wouldn’t receive the connected() signal from the first usage. This ensures that the signals received are safely coupled with the usage that caused them.

So how the heck do you retract a signal? By any means possible, I don’t care how you do it. Go digging in the event queue and destroy the event if you want. :) Personally, I tend to use QTimers with a timeout of 0, and emit my signal when the timeout() occurs. I use a real QTimer object, not singleShot(), so that I can call stop() on it if my object needs to reset, thus preventing the emit.

Here’s an example using QTimer:

class MyObject : public QObject
{
        Q_OBJECT
public:
        bool ready;
        QTimer readyTrigger;

        MyObject() : readyTrigger(this)
        {
                ready = false;
                connect(&readyTrigger, SIGNAL(timeout()),
                        SLOT(ready_timeout()));
                readyTrigger.setSingleShot(true);
        }

        // kicks the object into motion.  emit ready() when ready.
        void start()
        {
                ready = true;
                readyTrigger.start();
        }

        // reset the object to the initial state.  no ghost signals.
        void reset()
        {
                ready = false;
                readyTrigger.stop();
        }

signals:
        void ready();

private slots:
        void ready_timeout()
        {
                emit ready();
        }
};

There’s a little bit of delayed signals philosophy going on in there, if you’re confused about the way it is written.

Happy coding!

Comments

Nested Eventloops

To define the title: a nested eventloop is when you invoke the eventloop again rather than returning back to it. This is QCoreApplication::processEvents(), QDialog::exec (), QMessageBox::information(), and the like. In all but the most controlled situations, performing a nested eventloop using these traditional methods is dangerous and should be avoided. On the other hand, restricting event processing to a specific subset of objects can be safe. Unfortunately, Qt doesn’t provide a direct and general way to perform this kind of “scoped” eventloop.

First, let’s discuss why the traditional nested eventloop functions are bad. Spinning the eventloop is a big deal. Every object in the application (or, more specifically, in the thread, if your app is multithreaded, but most aren’t) will run when the eventloop is run. This means event handlers being called, possible signals emitted as a result of those events, etc. This can cause unpredictable behaviors. If the application has not “returned to the eventloop” yet, then there may be functions on the stack that are incomplete, and so the entire state of the program is not ready to receive events yet. A form of re-entrancy is also possible, whereby an event handler is called, sometime down the line the eventloop is run again, and then the same event handler is called again. The event handler is now on the stack twice. Did you write your application to work under this kind of condition?

Here is an example of an object that prompts the user for two socket ids, and then emits a signal when either one has data. This is a bit contrived, but it is short, and the problem should be easy to spot.

class MyObject
{
        QSocketNotifier *sn1, *sn2;

        void setup()
        {
                int sockfd1 = get_next_sockfd();
                sn1 = new QSocketNotifier(sockfd1, QSocketNotifier::Read);
                connect(sn1, SIGNAL(activated(int)), SLOT(sn_activated(int)));

                int sockfd2 = get_next_sockfd();
                sn2 = new QSocketNotifier(sockfd2, QSocketNotifier::Read);
                connect(sn2, SIGNAL(activated(int)), SLOT(sn_activated(int)));
        }

        int get_next_sockfd()
        {
                return QInputDialog::getInteger(0, "Prompt", "Socket FD:");
        }

signals:
        void dataReady();

private slots:
        void sn_activated(int)
        {
                delete sn1;
                delete sn2;
                emit dataReady();
        }
}

Do you see it? QInputDialog::getInteger() spins the eventloop. If the first socket has data to read, the second call to getInteger might cause sn_activated(sockfd1) to be called. If this happens, the program will likely crash at the “delete sn2;” line. Now, one obvious fix would be to set sn2 to zero at the top of setup(), but that’s not the point of this discussion. The point is that the use of getInteger() may mean you need to take additional precautions with your code, that you normally wouldn’t have to do. This was a simple example. In a more complex, real-world situation, you may not know which function is spinning a nested eventloop, and the fixes may not be as straightforward.

In simple applications, running a nested eventloop is not a big deal. If you aren’t running any non-GUI objects, then feel free to use QMessageBox::information(). It is convenient, and modality usually protects you from re-entrancy. However, if you do have non-GUI objects, then QMessageBox::information() is a potential disaster. Suppose you have slots listening for network data. Those slots might be called, and your application may not be written to work properly in such conditions.

Ultimately, what this means is that nested eventloops can be safe if you know they will be safe in a given situation. But if you don’t know that they will be safe, then you cannot use them. And if you are a library author, you’re in a pickle, because you cannot know if the application will be safe. Please, if you are writing a library, DO NOT use processEvents(), QMessageBox::information(), etc. If you absolutely must use a nested eventloop in a library, any method leading to that effect should be clearly documented. This way, the application developer can either avoid the function, or design his application to withstand the effects.

You might say that problems with nested eventloops are the result of poor programming at the higher level. That is, the application should be designed to “withstand the effects” of any method potentially running a nested eventloop. This is one of Trolltech’s positions (as of the time of this writing, October 23rd 2006, which I may talk more about later). Anyway, I’m here to tell you that you cannot write such a generic application. What if QString::operator+() invoked a nested eventloop? Good luck trying to make your application withstand such a thing, even if you were told about it up front. But having to account for any random method invoking a nested eventloop? Not possible. It would be like thread programming without mutexes.

We need assurances, as programmers, of what functions are going to do. We need to know what global variables they might modify (consider functions that return static data). We need to rely on their signatures being correct at runtime (return value, arguments). If we cannot have any expectations about the methods we call, then it would not be possible to write programs. If a method may run a nested eventloop, the user of the method must be informed about it.

In short, avoid using nested eventloops. This concept joins Signal Safety and Delayed Signals as another essential Qt programming practice for clean and predictable code.

Comments

In-place builds

It may not be widely known that Psi (as well as QCA and Qt) can be built and used “in-place”. This means you configure and make, but there is no ‘make install’ step. This can be convenient if you don’t want to install these packages system-wide.

Of course, you can always just set a prefix within your home directory (or other user-writable area) when building sources if you don’t wish to install system-wide. However, in-place builds are extra convenient for the developer, as it saves the trouble of having to do ‘make install’ every time something is changed.

So while this is not anything new for Psi development, and nor is the idea particularly revolutionary, it might be something you didn’t know you could do and so I’m documenting it here. Personally, I develop everything in-place.

Qt:

Qt 3 is/was always built in-place. You extract it where you want it to end up, configure and make. There is no ‘make install’ step. With Qt 4, they changed it to behave more like other packages. Now the prefix defaults to somewhere in /usr/local, and there’s a ‘make install’. However, the old way is still supported if you pass the directory of where you extracted Qt to -prefix. That is:

$ tar zxvf qt-x11-opensource-src-[version].tar.gz
$ cd qt-x11-opensource-src-[version]
$ ./configure -prefix $PWD
$ make

This is not a hack or a way to trick Qt. It is actually a supported feature, and when Qt is done configuring you’ll notice it says:

Qt is now configured for building. Just run 'make'.
Once everything is built, Qt is installed.
You should not run 'make install'.

Another feature of Qt 4 is that qmake is aware of the installation location. This allows it to locate the necessary files it needs without QTDIR. Additionally, all subtools (e.g. moc) are invoked using their full path. This means you don’t even need the Qt 4 bin directory in your PATH. Finally, if you’re using gcc, qmake also uses rpath when linking. This means you don’t need Qt 4′s lib directory to be specified anywhere either. No need for /etc/ld.so.conf or LD_LIBRARY_PATH. You simply build Qt, go into your project directory, and run qmake.

$ cd /path/to/myproject
$ /path/to/qt/bin/qmake
$ make

QCA:

Like Qt, QCA (as of 2.0 beta2, I believe) supports in-place builds by pointing prefix to $PWD.

$ ./configure --prefix=$PWD

Note: If you’re building against an in-place Qt, and you didn’t put Qt’s bin directory in your PATH, then you’re going to have to tell configure where Qt is. Do this with –qtdir. For example:

$ ./configure --prefix=$PWD --qtdir=/path/to/qt

Unlike Qt, QCA doesn’t use rpath. This may change in the future, but for now you will need to put QCA’s lib directory in your LD_LIBRARY_PATH if you want to use it in-place:

$ export LD_LIBRARY_PATH=$PWD/lib

There’s actually one hidden secret you must do next, and it is sort of cheating. You need to copy crypto.prf into your qmake feature directory. This is so projects that need QCA can find QCA.

$ cp crypto.prf /path/to/qt/mkspecs/features

Fortunately, this file doesn’t ever change, so if you later modify QCA in-place, there’s no need to copy this file again.

QConf:

With the latest development version (as of October 22nd, 2006), qconf can be run without doing ‘make install’. In fact, you can run it from another location, and it locates its data files relative to the executable. This allows it to work similar to qmake, in that you can invoke it by specifying the full path. Unlike qmake, however, this is not dependent on the prefix. That is, you can configure qconf with the default prefix of /usr/local, and you can still run qconf from anywhere without installing it.

$ ./configure --qtdir=/path/to/qt
$ make
$ cd /path/to/myproject
$ /path/to/qconf
$ ./configure ...

Psi:

Psi can be built and used in-place. Once compiled, just run it.

$ ./configure --qtdir=/path/to/qt
$ make
$ ./psi

That’s it!

You can build and use all the Psi tools without mucking with your system, and without ever doing ‘make install’ either.

Comments (2)

invokeMethodWithVariants

QMetaObject::invokeMethod uses QArgument to store arguments and return values. But wouldn’t it be cool if we could use QVariant instead of QArgument? QArgument is just a thin wrapper around a pointer, and using it in a signal is certainly dangerous. On the other hand, putting argument values inside QVariants means we would have real objects that we can marshall around, which would give us great flexibility in performing dynamic function calls.

We already talked about how to dynamically obtain return types. The getReturnType function, as well as what we learned from it, should allow us to make a function for invoking methods that uses QVariant instead of QArgument.

Okay, so here’s getReturnType:

static QByteArray getReturnType(const QMetaObject *obj,
        const QByteArray &method, const QList argTypes)
{
        for(int n = 0; n < obj->methodCount(); ++n)
        {
                QMetaMethod m = obj->method(n);
                QByteArray sig = m.signature();
                int offset = sig.indexOf('(');
                if(offset == -1)
                        continue;
                QByteArray name = sig.mid(0, offset);
                if(name != method)
                        continue;
                if(m.parameterTypes() != argTypes)
                        continue;

                return m.typeName();
        }
        return QByteArray();
}

Now for the new stuff:

bool invokeMethodWithVariants(QObject *obj,
        const QByteArray &method, const QVariantList &args,
        QVariant *ret, Qt::ConnectionType type = Qt::AutoConnection)
{
        // QMetaObject::invokeMethod() has a 10 argument maximum
        if(args.count() > 10)
                return false;

        QList argTypes;
        for(int n = 0; n < args.count(); ++n)
                argTypes += args[n].typeName();

        // get return type
        int metatype = 0;
        QByteArray retTypeName = getReturnType(obj->metaObject(),
                method, argTypes);
        if(!retTypeName.isEmpty())
        {
                metatype = QMetaType::type(retTypeName.data());
                if(metatype == 0) // lookup failed
                        return false;
        }

        QGenericArgument arg[10];
        for(int n = 0; n < args.count(); ++n)
                arg[n] = QGenericArgument(args[n].typeName(),
                        args[n].constData());

        QGenericReturnArgument retarg;
        QVariant retval;
        if(metatype != 0)
        {
                retval = QVariant(metatype, (const void *)0);
                retarg = QGenericReturnArgument(retval.typeName(),
                        retval.data());
        }

        if(!QMetaObject::invokeMethod(obj, method.data(), type, retarg,
                arg[0], arg[1], arg[2], arg[3], arg[4],
                arg[5], arg[6], arg[7], arg[8], arg[9]))
        {
                return false;
        }

        if(retval.isValid() && ret)
                *ret = retval;
        return true;
}

Now let's compare QMetaObject::invokeMethod to invokeMethodWithVariants. Well, here's the example from invokeMethod's documentation:


QString retVal;
QMetaObject::invokeMethod(obj, "compute", Qt::DirectConnection,
        Q_RETURN_ARG(QString, retVal),
        Q_ARG(QString, "sqrt"),
        Q_ARG(int, 42),
        Q_ARG(double, 9.7));

Let's call the same function, using invokeMethodWithVariants:

QVariantList args;
QVariant retVal;
args << "sqrt" << 42 << 9.7;
invokeMethodWithVariants(obj, "compute", args, &retVal,
  Qt::DirectConnection);
// assuming successful invoke, retVal should contain a QString

Now you can call any method dynamically. Yay!

This trick is used in Leapfrog's Qt-to-Cocoa object bridge. We'll also make use of it in the Synchronized Threads series.

Comments (1)

Delayed Signals

Delayed signals are a design mechanism for reducing errors by people using your object (similar in spirit to Signal-Safety). This is a difficult topic, because up until this writing I still haven’t been able to formally define a universal rule for when you should use delayed signals. I’ll attempt that now, and show some code to illustrate.

Rule: When requesting a result from an object by first calling a member function and then obtaining the result via a signal, the signal should not be emitted until after the application has returned to the event loop.

To explain just what the heck that means, consider how signal usage can fall into one of two groups: 1) as a means of returning or indicating a result of a requested operation, or 2) as a means of announcing general status changes, independent of a request. #1 is used in worker objects, for example QTcpSocket or QProcess, where there is a request/response expectation. #2 is used predominantly in GUI objects (widgets), where the signals that occur are not requested nor required to operate.

My rule only pertains to #1. One good way to ask yourself if your signal falls under the rule is if it appears to play a part in returning an answer. Synchronous methods will use the return value to deliver an answer. Asynchronous methods don’t use the return value for the answer, and instead give or indicate the answer via a signal. If this is you, then you should use delayed signals.

Now, with that out of the way, you might ask: why use delayed signals?

Well, first of all, you probably wouldn’t be using a signal if you didn’t expect your response to take time to generate. If your object uses the event loop to do its processing before signalling a result, then your signals will already be “delayed” in the sense I’m talking about.

The problem arises when your object knows the result early, and therefore the object does not need to run the event loop in order to generate the result. If the signal is emitted before returning to the event loop (for example, if it is emitted during the “request” member function), then this would not qualify as delayed.

If your object is unpredictable, that is it sometimes emits delayed and other times non-delayed, then the user of your object may have to write extra code to handle both possibilities. The whole point of using a signal is to be able to provide an asynchronous reply, so if we want to pick just one pattern then “delayed” is our answer. It is the more natural pattern for signals anyway.

If you aren’t convinced, the best explanation is with code. Suppose there’s an object that downloads a file from a URL:

class Downloader : public QObject
{
        Q_OBJECT
public:
        void start(const QString &url);
        void stop();

signals:
        void finished(bool success);
};

Simple enough. It will emit finished(true) if the file download succeeded, or finished(false) if there is a problem.

Now let’s make a simple app that tries to download two files, and immediately quits if there is a problem:

class App : public QObject
{
        Q_OBJECT
public:
        Downloader file1, file2;
        int done;

        void start()
        {
                connect(&file1, SIGNAL(finished(bool)), SLOT(dl_finished(bool)));
                connect(&file2, SIGNAL(finished(bool)), SLOT(dl_finished(bool)));

                printf("Starting downloads\\n");
                done = 0;
                file1.start("http://example.com/document.txt");
                file2.start("http://example.com/otherDocument.txt");
        }

signals:
        void quit();

private slots:
        void dl_finished(bool success)
        {
                if(success)
                {
                        ++done;
                        if(done == 2)
                        {
                                printf("Downloads success.\\n");
                                emit quit();
                        }
                }
                else
                {
                        file1.stop();
                        file2.stop();

                        printf("Error during downloads!\\n");
                        emit quit();
                }
        }
};

Looks good, yes?

Actually, no, it isn’t quite right. We assumed that signals would always be emitted to us delayed. If Downloader doesn’t emit delayed 100% of the time, then we have bugs.

Let’s assume Downloader may or may not always emit delayed. We’ll write extra code then, to handle all situations:

class App : public QObject
{
        Q_OBJECT
public:
        Downloader file1, file2;
        int done;
        bool error;

        void start()
        {
                connect(&file1, SIGNAL(finished(bool)), SLOT(dl_finished(bool)));
                connect(&file2, SIGNAL(finished(bool)), SLOT(dl_finished(bool)));

                printf("Starting downloads\\n");
                done = 0;
                error = false;
                file1.start("http://example.com/document.txt");
                if(!error)
                        file2.start("http://example.com/otherDocument.txt");
        }

signals:
        void quit();

private slots:
        void dl_finished(bool success)
        {
                if(success)
                {
                        ++done;
                        if(done == 2)
                        {
                                printf("Downloads success.\\n");
                                emit quit();
                        }
                }
                else
                {
                        file1.stop();
                        file2.stop();
                        error = true;

                        printf("Error during downloads!\\n");
                        emit quit();
                        QTimer::singleShot(0, this, SIGNAL(quit()));
                }
        }
};

If finished(false) is emitted non-delayed by file1.start(), then this means we will perform our deinitialization process before App::start() completes. We use the ‘error’ bool to track if this happens, so that we can skip the calling of file2.start() as necessary. Since file2.stop() would occur before the call to file2.start() (had we not fixed the code), this would most likely mean that the second file transfer wouldn’t be stopped at all.

Finally, if we do a normal emit of quit() during a non-delayed call to finished(), then we lose signal-safety. If App is deleted during the slot that listens to quit(), the code following file1.start() will cause a crash when App’s members are accessed (such as the ‘error’ bool). We can give ourselves signal-safety by doing a delayed emit of quit() with QTimer::singleShot.

Bottom line: there’s less code if we can assume that Downloader always uses delayed signals.

Comments (3)

Dynamic return types

QMetaObject::invokeMethod allows calling methods on Qt objects dynamically. The method name is passed as a string, and the arguments are put together with QGenericArgument. If you haven’t read the documentation for the invokeMethod function, do so. It will open your eyes about Qt.

There’s just one problem: how do you handle a dynamic return value? QGenericReturnValue has to be set in advance with the type of value being returned. If we want to have a fully dynamic call, with even the return value type determined at runtime, then we need a way of inspecting a method to obtain its return type before we call it. After that, we can set up our QGenericReturnValue properly and call the method.

So, how do you get the return value type? Like this:

QByteArray getReturnType(const QObject *obj,
        const QByteArray &method,
        const QList argTypes)
{
        const QMetaObject *mo = obj->metaObject();
        for(int n = 0; n < mo->methodCount(); ++n)
        {
                QMetaMethod m = mo->method(n);
                QByteArray sig = m.signature();
                int n = sig.indexOf('(');
                if(n == -1)
                        continue;
                QByteArray name = sig.mid(0, n);
                if(name != method)
                        continue;
                if(m.parameterTypes() != argTypes)
                        continue;

                return m.typeName();
        }
        return QByteArray();
}

Now you can have some code:


class MyObject : public QObject
{
        Q_OBJECT
        ...
public slots:
        QString intToString(int x)
        {
                return QString::number(x);
        }
};

MyObject *obj = ...
QByteArray methodName = "intToString";
QList argTypes;
argTypes += "int";
QByteArray retType = getReturnType(obj, methodName, argTypes);
// the value of retType is now "QString"

At this point, we have the method name, the argument types, and the return type. For homework: figure out how to use this information in order to make an invokeMethod() call. I’ll explain how in a later article.

Comments

Synchronized Threads (Part 2)

The astute observer will notice a problem in the code I provided for Synchronized Threads (Part 1), which is that the eventloop begins only after we unlock the mutex.

Why is that bad? Well, this means that the calling thread will resume before the eventloop in the SyncThread has started. If the calling thread then calls stop() right away, it is possible that the “quit” method of the QEventLoop object will be delivered too early. If this happens, then the eventloop won’t actually quit, and the calling thread will be stuck forever. Additionally, code within atStart() occurs without an eventloop running. These two problems are nearly identical to those covered in the startmyappLater article, which is why I published it before this one.

Correcting this problem in SyncThread takes a little bit of effort. We are going to need a helper QObject that runs in our thread, which I’ve called SyncThreadAgent.

class SyncThread : public QThread
{
        Q_OBJECT
private:
        QMutex m;
        QWaitCondition w;
        QEventLoop *loop;
        SyncThreadAgent *agent;

public:
        SyncThread(QObject *parent = 0);
        ~SyncThread();

        void start();
        void stop();

protected:
        virtual void run();
        virtual void atStart() = 0;
        virtual void atEnd() = 0;

private slots:
        void agent_started();
};

class SyncThreadAgent : public QObject
{
        Q_OBJECT
public:
        SyncThreadAgent(QObject *parent = 0) : QObject(parent)
        {
                QMetaObject::invokeMethod(this, "started", Qt::QueuedConnection);
        }

signals:
        void started();
};

SyncThread::SyncThread(QObject *parent)
:QThread(parent)
{
        loop = 0;
        agent = 0;
}

SyncThread::~SyncThread()
{
        stop();
}

void SyncThread::start()
{
        QMutexLocker locker(&m);
        Q_ASSERT(!loop);
        QThread::start();
        w.wait(&m);
}

void SyncThread::stop()
{
        QMutexLocker locker(&m);
        if(!loop)
                return;
        QMetaObject::invokeMethod(loop, "quit");
        w.wait(&m);
        wait();
}

void SyncThread::run()
{
        m.lock();
        loop = new QEventLoop;
        atStart();
        w.wakeOne();
        m.unlock();
        agent = new SyncThreadAgent;
        connect(agent, SIGNAL(started()), SLOT(agent_started()), Qt::DirectConnection);
        loop->exec ();
        m.lock();
        atEnd();
        delete agent;
        agent = 0;
        delete loop;
        loop = 0;
        w.wakeOne();
        m.unlock();
}

void SyncThread::agent_started()
{
        atStart();
        w.wakeOne();
        m.unlock();
}

Now everything should be correct.

Stay tuned, we will do more with SyncThreadAgent in Part 3.

Comments

« Previous Page« Previous entries « Previous Page · Next Page » Next entries »Next Page »