Log files
Where they are, and what they are called
An appender writes one file per day into the log directory of its config:
<logDir>/<namePrefix>_YYYYMMDD.xlogXlogConfig(logDir = "/data/…/xlog/log", namePrefix = "marsrs") gives marsrs_20260927.xlog. The directory is created when it is not there, and the prefix is what the appender is known by: two appenders that share one share the file, and closing one of them closes what the other writes through. Give a part of an app whose logs are read apart from the rest a prefix of its own.
Where the appender writes:
xlog.current_log_path() // Option<PathBuf>, and it is the directorylog.currentLogPathchar path[512];
mars_xlog_current_log_path_instance(handle, path, sizeof path); // bytes, or a negative codexlog.currentLogPathxlog.currentLogPathxlog.currentLogPathawait xlog.currentLogPath()xlog.currentLogPathEvery package here answers it, and every one of them answers it about the Xlog the file belongs to — and what every one of them answers is the directory, which is what the C++'s GetCurrentLogPath hands back. The day's file is the question under it, so between the two an app never has to put a day's name together itself.
A whole day of files is what an app that uploads yesterday's asks for, and there are two calls for it: the files that are there, and the names the day is written under whether or not they are there yet.
xlog.log_files(1) // yesterday's, that are there
xlog.log_file_names(1) // the names, whether or notlog.logFiles(daysAgo: 1)
log.logFileNames(daysAgo: 1)char path[512];
// index 0, 1, 2 …; a negative code — MARS_XLOG_ERR_NO_PATH — is the end of the list
mars_xlog_getfilepath_from_timespan_instance(handle, 1, 0, path, sizeof path);
mars_xlog_make_logfile_name_instance(handle, 1, 0, path, sizeof path);xlog.logFiles(1L)
xlog.logFileNames(1L)xlog.logFiles(1L)
xlog.logFileNames(1L)xlog.logFiles(1)
xlog.logFileNames(1)await xlog.logFiles(1)
await xlog.logFileNames(1)xlog.logFiles(1)
xlog.logFileNames(1)0 is today and 1 is yesterday. Rust and Swift answer the day's list in one call, and out of the Xlog the day belongs to; the C ABI answers one index of it at a time, and the two C calls are that walk. Names answer two where files answer one when a cache dir is given and the file is there: the log-dir file and its twin in the cache dir.
Async: the record may still be in the cache
The default mode hands a record to a writer thread through a memory-mapped cache file, so a write returns before the bytes reach the log file. The cache lives in cacheDir, or next to the log files when you give none.
Flush before you read or upload — this process reads the file, or another one reads it while this one is still logging into it:
xlog.flush_now() // waits for the filelog.flushNow()xlog.flushNow()xlog.flushNow()xlog.flushNow()await xlog.flush()xlog.flushNow()mars_xlog_flush_now_instance(handle);The drain is three calls and not one with a flag:
| call | what it does |
|---|---|
requestFlush() — xlog.request_flush(), mars_xlog_request_flush_instance(handle) | asks for the drain and returns at once: nothing answers when it is over, and the records are not in the file when it returns |
flushNow() — xlog.flush_now(), mars_xlog_flush_now_instance(handle) | drains on the calling thread: the records are on disk when it returns |
await flush() — xlog.flush().await | the same drain, handed to another thread: it answers when the records are on disk |
requestFlush() is the one a timer calls: it asks for the drain and answers nothing about when it is over, and nothing is lost while it is not, because a record still in the cache is in a file the kernel holds. flushNow() is the one to call before the file is read or uploaded, and await flush() is that same drain for a caller that would rather not hold a thread. Rust carries all three, and its await flush() is a Future written against std::thread::spawn — no runtime, and any executor waits on it. Dart has no flushNow(), because a method channel cannot block the Dart side of it, and HarmonyOS has no await flush(), because every method of its NAPI module is synchronous.
When the app goes away
There is nothing to call. Nothing is lost when a process is killed mid-write, either: a record in the cache is in a file the kernel holds and not in the process, so it outlives the process, and the next appender of the same namePrefix drains it into its log file as it opens — between the ~~~~~ begin of mmap ~~~~~ and ~~~~~ end of mmap ~~~~~ lines of an ordinary start. What an app that never flushes gives up is only time: the last records of a session reach its file when the next one begins.
Two of the packages give up nothing at all, because they flush when the app leaves the screen — the last moment Android and iOS say anything before they can end the process without another word:
| where | what flushes it |
|---|---|
| Android | Xlog(config, context) — any Context of the app registers a ComponentCallbacks2 that flushes from TRIM_MEMORY_UI_HIDDEN up |
| SwiftPM | every Xlog, from the moment it is built: it watches didEnterBackground and willTerminate, and WKExtension's on watchOS |
| everywhere else | the next start, as above |
close() drains too, so an app that closes its appender on the way out is covered by that as well. The flush above is for the other case: a read or an upload that happens while the app is still running.
Sync mode is the one exception, and it is the one place the answer is not "nothing": there is no cache file behind its records, so the tail the process was still holding — up to about 4 KiB of it — dies with the process. An app that logs synchronously and wants that tail has to close() or flushNow(); the two hooks above do it, and so does an app that closes its appender.
Rotation and retention
| knob | what it does | default |
|---|---|---|
maxFileSizeBytes | a file is closed and a new one opened once it reaches this many bytes | 0 — never split |
maxAliveTimeSeconds | a file older than this many seconds is dropped | 0 — ten days; a value under a day is refused, and the one already in force stands |
cacheDays | how many days a file staged in the cache directory waits before it is moved into the log directory | 0 — nothing is staged: the day's file is written in the log directory |
Reading a file back
A .xlog written here is the .xlog the C++ implementation writes — same framing, same compression, same encryption — so the tooling that already reads mars logs reads these.
# The crate is not on crates.io yet — publication is pending — so this is the
# tag; a release carries the same command as an archive.
cargo install --git https://github.com/orangeboyChen/mars-rs --tag v0.1.0-alpha.3 marsrs-xlog
xlog decode --privkey=<hex> marsrs_20260927.xlog --out=marsrs.plainuse marsrs::xlog::{get_period_logs, LogBuffer};
// the bytes of the records between two hours of the day
let (begin, end) = get_period_logs(std::path::Path::new("marsrs_20260927.xlog"), 0, 24)?;python3 decode_mars_log_file.py marsrs_20260927.xlog # Tencent/marsAn encrypted file needs the private key of the pair whose public key is in the config — xlog decode takes it as --privkey, and nothing reads the records without it. xlog encode is the other half: it writes a .xlog out of one record per line of its input, and encrypts it when it is given the public key of that pair with --pubkey.
A file that is not whole still reads. A record that cannot be read — a block a process killed between two writes never finished, a byte that went wrong on its way to wherever the file was copied — is skipped, and the span it took is marked in the output where that record's text would have been, so the records behind the damage are in what the CLI writes out and not lost with it.
The CLI page is the whole command line — installing it, making that pair, reading a file back and writing one. xlog help prints the same thing in a terminal, and configuration says what the public key does to a record.