Dovecot dsync backup examples how to#

if backend doesn't support locking or it's slow, single-dbox should be used (instead of multi-dbox), because it doesn't need locking.ДЄsync input streams are created with FS API handle, so it's possible to start reading from them before the file has even been open()ed.

this isn't an asynchronous operation! it assumes that locking state is kept in memory, so that the operation will be fast.

if file was created and not committed, the temp file will be deleted.

mdbox saving can write many temp files in a single transaction and only at commit stage it locks the index files and knows what the filenames will be.

if filename is given and a new file is being created, the filename is changed to the given one instead of using the original path's filename.

no writes will be visible to reads until commit() is called.

data can only be appended to files, there is no support for overwriting data. works in a similar way than writing to socket.

append data to given file and return how many bytes were actually added to write buffer.

this would require memcpy()ing all data, but it might get too complex/fragile if it was asynchronously written to given buffer. so the idea is that the backend implementation would buffer/readahead data, which would be returned by this call.

just like pread(), but can fail with EAGAIN if there are no bytes already buffered.

call the callback when more data can be read/written.

Set_output_callback(handle, callback, context) Set_input_callback(handle, callback, context) but it should contain mode and gid in some format

permissions: i'm not entirely sure how this works yet.

mode=replace-if-exists: commit() replaces file if it already exists.

mode=fail-if-exists: commit() fails if file already exists.

mode=append: appending to file is allowed.

Non-blocking I/O works pretty nicely for socket I/O, so I thought I'd use a similar API for disk I/O as well: The idea is to first abstract out all POSIX filesystem accessing in dbox/mdbox code. The target Dovecot version is v2.1, not v2.0. I originally thought about writing a more specific plan, but I think this is good enough now to start coding.

Dovecot dsync backup examples how to#

Luckily all of that already supports nonblocking input or can be easily modified to support it.Д«elow are some thoughts how to go forward with this. It would also be used internally by all of message parsing/searching code. Whenever sending message data to client, this stream would have to be used. The returned stream can return EAGAIN whenever it would need to block. Then again, the API changes are necessary, so the parallelism optimizations probably aren't a huge task on top of that.Д«esides the API changes described in the above link, another change that's necessary is to add mail_get_nonblocking_stream(). Adding support for all of the paralellism would be nice, but probably not necessary as the first step. I already started describing how the lib-storage API could be changed to support high-latency storages. This might also allow mbox/maildir to become partially asynchronous. The first step to that direction would be to make dbox/mdbox I/O asynchronous. The long term plan is to get all of Dovecot disk I/O asynchronous.