zzzip/docs/API.md

ZZZip Archive Library API

Include zzz.h, and link with -lzzz.

You can compile the library with creation support only using ZZZ_CREATEONLY=1 environment variable, and extraction only library with the ZZZ_EXTRACTONLY=1 environment variable to make. When neither of those environment variables set, the library will include both archive creation and extraction functions.

[[TOC]]

Error Codes

The functions are clearly named and have straightforward arguments. They return an integer error code which is defined as follows:

Define	Description
ZZZ_OK	success (0)
ZZZ_ENCRYPTED	returned on extraction, when the archive is encrypted and requires password
ZZZ_END	returned on extraction, when the end of the archive or data is reached
ZZZ_ERR_BADINP	bad input parameters
ZZZ_ERR_NOMEM	memory allocation error
ZZZ_ERR_HDRTOOBIG	header is too big, too many entity attributes
ZZZ_ERR_IO	File I/O error when reading or writing the archive
ZZZ_ERR_CORRUPT	File format error or bad CRC checksum
ZZZ_ERR_UNSUPPORTED	Unsupported filter, encryption or compression method in the archive

Creating Archives

Only available if compiled without ZZZ_EXTRACTONLY. The scheme goes like this:

1. call a function to create an archive
2. add an entity to the archive
3. add extra information for the entity (optional)
4. add entity data to the archive (optional)
5. call entity flush
6. repeat steps 2 - 5 for all the entities you want to store in the archive
7. call finish to finalize the archive

The API was designed to support non-seekable pipe and network streams too, however that's not always possible, see entity data below.

Create Archive on Disk

void *zzz_create_file(char *fn);

Creates an archive file on disk.

Create an Archive Stream

void *zzz_create_stream(int fd);

Creates an archive stream.

Create Archive in Memory

void *zzz_create_mem();

Creates an archive in memory.

Set Compression

int zzz_compression(void *ctx, int method, int level);

Sets compression level for compressing the entire archive (filenames and attributes will be compressed too, like in tar and cpio). Currently only ZZZ_FILTER_ZSTD method supported, level can be 1 to 22, 5 or 6 is a good choice (zstd uses a reference table indexed by this level to select the actual method and compression parameters). If you call this, it must be called right after zzz_create_file, zzz_create_stream, or zzz_create_mem. If there's no call to this function, then entities will be individually compressed (like in zip and rar).

Set Encryption

int zzz_encrypt(void *ctx, int method, uint8_t *password, int pwdlen);

Add a method to the encryption chain. This function might be called multiple times for strengthening security. Available methods are ZZZ_ENC_AES_256_CBC (AES256) and ZZZ_ENC_SHA_256_XOR (a simple symmetric XOR cipher). Note that the latter is a symmetric cipher, which means if you call it twice in a row, then there will be no encryption! It's pretty strong in itself as it uses multiple masks, but it's main purpose is to use in conjunction with AES. The password and pwdlen arguments must be the same for all calls within one archive.

Adding an Entity to the Archive

File

int zzz_entity_file(void *ctx, char *fn, int text, int64_t size);

Adds a file entity to the archive. The filename in fn must be UTF-8 encoded, using / as directory separator. If text is true (or ZZZ_TEXT), then the file contents will be threated as text/plain, and OS-specific line endings will be replaced by '\n' characters. Otherwise (or when text is ZZZ_BINARY) the file is assumed a binary file.

This call should be followed by zzz_entity_data calls. For text files, only one call allowed, for binaries data can be added in multiple rounds.

Links

int zzz_entity_hardlink(void *ctx, char *fn, char *target);
int zzz_entity_symlink(void *ctx, char *fn, char *target);

Hard link entity's target must match one of the previous file entity's fn. For symbolic links the target can be anything. Target is a zero terminated UTF-8 string.

Directory

int zzz_entity_dir(void *ctx, char *fn);

Adds a directory to the archive. This only adds the directory record, and does not walk the directory to add its sub-folders, that must be implemented in the application using the library.

Directory Union

int zzz_entity_union(void *ctx, char *fn);
int zzz_entity_union_target(void *ctx, char *target);

Adds a directory union to the archive. The zzz_entity_union_target function might be called multiple times. Directory unions are symbolic links like constructs. This only makes sense on special OSes.

Device Files

int zzz_entity_chrdev(void *ctx, char *fn, uint32_t devmaj, uint32_t devmin);
int zzz_entity_blkdev(void *ctx, char *fn, uint32_t devmaj, uint32_t devmin);

Adds a character or block device file to the archive. This is only makes sense on certain OSes, mostly on POSIX ones.

Named FIFO

int zzz_entity_fifo(void *ctx, char *fn);

Adds a named pipe (FIFO) to the archive. This is only makes sense on certain OSes, mostly on POSIX ones.

Setting Entity's Attributes

Calling these function always set the attribute for the entity last added to the archive.

Last Modification Date and Time

int zzz_entity_mtime(void *ctx, uint16_t year, uint8_t mon, uint8_t day, uint8_t hour, uint8_t min, uint8_t sec);
int zzz_entity_mtime_t(void *ctx, uint64_t t);

Sets the OS-independent last modification date and time field. The first is always available, the second one, which accepts a UNIX timestamp as input only on OSes which support gmtime().

See also zzz_entity_extra_time for nanosec precision timestamps.

Comment

int zzz_entity_extra_comment(void *ctx, char *comment);

Adds a zero terminated, UTF-8 comment to the entity.

Mime Type

int zzz_entity_extra_mime(void *ctx, char *mime);

Sets the entity's mime type. The mime argument must be in "(main)/(sub)" format, like text/html, and it might contain an additional ";charset=X" argument.

Icon

int zzz_entity_extra_icon(void *ctx, uint8_t *buf, uint16_t size);

Icon should be small (about 32k tops). Any format can be used as long as the image's format can be detected by magic bytes, however PNG and SVG is strongly preferred for interoperability.

Meta Information (XAttr)

int zzz_entity_extra_meta(void *ctx, char *key, uint8_t *value, uint16_t size);

Adds a meta information with key to the entity. This function can be called multiple times.

Separate Timestamps

int zzz_entity_extra_time(void *ctx, uint64_t mtime, uint64_t atime, uint64_t ctime, uint64_t btime);

Sets specific time fields for the entity in nanosec precision. mtime is the modification time, atime is the last access time, ctime is the last status change time, and btime is the file creation (birth) time. Except the modification time, all the other timestamps are optional.

UNIX Access Rights

int zzz_entity_extra_access(void *ctx, uint32_t mode, char *user, uint64_t uid, char *group, uint64_t gid);

Sets the UNIX access rights of an entity. Both user and group must be a zero terminated UTF-8 string.

Access Control Lists

int zzz_entity_extra_acl_nfs(void *ctx, char *acl);
int zzz_entity_extra_acl_text(void *ctx, char *acl);
int zzz_entity_extra_acl_uuid(void *ctx, uint8_t *acl, uint16_t size);

These functions are designed that they can add an entire list at once, or they could be called multiple times as well to add ACEs one-by-one. zzz_entity_extra_acl_nfs is used to store POSIX NFSv4 ACL, in the format "type:flags:principal:permissions\n". zzz_entity_extra_acl_text are newline terminated, free-form ACLs, used for OS/2 ACLs and POSIX 1003e draft standard 17 ACLs. zzz_entity_extra_acl_uuid uses UUIDs, where the access rights are stored in the last byte of the UUID, size must be multiple of 16. WindowsNT ACLs are stored as OS-specific extra field, see adding arbitrary attributes below.

OS-Specific Attributes

int zzz_entity_extra_os_tag(void *ctx, int ostype, uint16_t tag, uint8_t *buf, uint16_t size);

Other OS-specific data can be embdedded using a pre-defined ostype magic value and an OS-specific tag key, if the data is organized in a key-value pairs.

The full list of possible ostype values can be found in the ZZZip file format specification's Appendix.

Arbitrary Attributes

int zzz_entity_extra(void *ctx, int type, uint8_t *buf, uint16_t size);

This function is used to add any attribute otherwise not covered by the API. WindowsNT ACLs are stored with type being ZZZ_EXTRA_OS_WIN, and buffer containing a binary ACL structure followed by DWORD aligned ACE entries.

Adding Entity Data to Archive

int zzz_entity_data(void *ctx, uint8_t *buf, uint64_t size);

Adds data to the archive for the last added entity. For text files, this can be only called once, for binary files there might be multiple calls adding arbitrary amount of data at once. Only for file entities, others do not have contents.

When generating archive to a stream, entire archive compression must be set, see zzz_compression above, otherwise you're limited to one zzz_entity_data call per entity for binary files too.

Flushing Entity to Archive

int zzz_entity_flush(void *ctx);

When all attributes are set, and all data is written into the archive for an entity, this function must be called.

Finalizing the Archive

int zzz_finish(void *ctx, uint8_t **mem, uint64_t *size);

As a last step, this function closes the archive. When it was created with zzz_create_mem, then the mem pointer returns a buffer to the compressed archive. It can be NULL. If an integer pointer is given in size, then the size of the compressed archive is returned too (no matter how the archive was created). When the archive was created into a stream, fd is NOT closed.

Extracting Archives

Only available if compiled without ZZZ_CREATEONLY. The scheme goes like this:

1. call a function to open an archive
2. in a loop, read entities from the archive
3. if read returns ZZZ_ENCRYPTED, then ask the user for password, goto 2
4. if read returns ZZZ_OK, and it is a file, read file contents, goto 2
5. if read returns ZZZ_END, then there are no more entities in the archive
6. close the archive

The API was designed to support non-seekable pipe and network streams too, meaning you're expected to read through the archive in one pass from beginning to end. It supports both entire compressed archives and per entity compressed archives. The zlib compressed archives with RFC1950 gzip header can only be read from seekable files or from memory and they are limited to 4G in size. Instead bzip2, xz, and zstd can be used (archive creation uses zstd by default).

Open Archive from Disk

void *zzz_open_file(char *fn);

Opens an archive on disk for reading. The fn argument must be a zero-terminated UTF-8 string.

Open Archive in a Stream

void *zzz_open_stream(int fd);

Opens an archive in a stream for reading. Does not support zlib compressed archives with gzip header, all the other compression methods can be used.

Open Archive in Memory

void *zzz_open_mem(uint8_t *mem, uint64_t size);

Opens an archive in a memory buffer for reading.

Decrypt Archive

int zzz_decrypt(void *ctx, uint8_t *password, int pwdlen);

Sets decryption password for accessing the archive. Should be called when reading entity returned ZZZ_ENCRYPTED.

Read Entity from Archive

int zzz_read_header(void *ctx, zzz_entity_t **ent);

Returns a pointer with the entity's name and metadata. This should be called multiple times until it returns ZZZ_END or one of the error codes. After each iteration, the next entity's data can be accessed in ent, but that pointer is only valid if the function returns ZZZ_OK. If it returns ZZZ_ENCRYPTED or ZZZ_END, then ent is unset. No need to free this entity structure, the library takes care of that. This also means if you need one of its data permanently, you must copy it out before the next zzz_read_header call.

The entity's type can be queried by ZZZ_FILE_TYPE(ent->header.type). It can be one of the following:

Define	Description
ZZZ_TYPE_REG	a regular file (can be plain text or binary)
ZZZ_TYPE_LNK	a hard link
ZZZ_TYPE_SYM	a symbolic link
ZZZ_TYPE_UNI	a directory union
ZZZ_TYPE_CHR	a character device
ZZZ_TYPE_BLK	a block device
ZZZ_TYPE_DIR	a directory
ZZZ_TYPE_FIFO	a named pipe

Other fields of the entity structure:

zzz_entity_t	Description
header.myear	last modification year
header.mmon	last modification month 1 - 12
header.mday	last modification day 1 - 31
header.mhour	last modification hour 0 - 23
header.mmin	last modification minute 0 - 59
header.msec	last modification second 0 - 60
header.type	entity's type, use with ZZZ_FILE_TYPE(x) macro, see above
header.uncompressed	uncompressed file size
header.contentsize	length of the target field (only for links and unions)
header.namelen	file name's total length, including zero terminator character
filename	file name, zero terminated UTF-8
target	target for links and unions
devmaj, devmin	device major and minor numbers for device files
numextra	number of extra attributes
extra	an array of extra attributes, in zzz_attr_t structure

zzz_attr_t	Description
type	attribute's type
size	attribute's size (use with attr.any)
rev	attribute's format revision (OS-specific only, see note)
attr.time.mtime	last modification timestamp (if type ZZZ_EXTRA_TIME)
attr.time.atime	last access timestamp (if type ZZZ_EXTRA_TIME)
attr.time.ctime	last statis change timestamp (if type ZZZ_EXTRA_TIME)
attr.time.btime	file creation (birth) timestamp (if type ZZZ_EXTRA_TIME)
attr.access.mode	access mode bits (if type ZZZ_EXTRA_ACCESS)
attr.access.uid	numeric user id (if type ZZZ_EXTRA_ACCESS)
attr.access.user	username, zero terminated string (if type ZZZ_EXTRA_ACCESS)
attr.access.gid	numeric group id (if type ZZZ_EXTRA_ACCESS)
attr.access.group	groupname, zero terminated string (if type ZZZ_EXTRA_ACCESS)
attr.meta.len	number of meta values (if type ZZZ_EXTRA_META)
attr.meta.data	meta key value pairs (if type ZZZ_EXTRA_META)
attr.any	unsigned char buffer for accessing all the other types

rev - most significant bit indicates that the OS-specific data in attr.any is in big-endian format. Only valid if type >= 16, ZZZ_EXTRA_OS_OS2.

Timestamps, mode, numeric ids and all other integer values are automatically converted from little-endian to host-native endian values, except for OS-specific attributes. Those must be parsed by the application, not interpreted by the library in any way.

Read Entity Data from Archive

int zzz_read_data(void *ctx, uint8_t *buf, uint64_t *size);

This function must be called if the returned entity's header.uncompressed size is not zero. This function might be called multiple times, using arbitrary sized buffers. Seek is not possible, data must be read otherwise entity CRC's validity can't be cheched.

If the entity is a text file, then line endings will be converted to OS native line endings. Because that might be more than one character on certain OSes, for UNIX based OSes the value in header.uncompressed might be bigger than the actual uncompressed size. This function returns ZZZ_OK when there's more data to read, and ZZZ_END when there's no more data. For binary files, header.uncompressed is always exactly the same as the size of the returned data.

Close the Archive

int zzz_close(void *ctx);

As a final step, the archive must be closed to free the context. When the archive was read from a stream, fd is NOT closed.