Detailed Description

File handle pool and simple file operations e.g.

FileOffset offset types:

copy and delete.

Introduction

Implements a pool of file handles for use within multi-threaded environments where the overhead of opening and closing a file handle for each thread would be too great. Primarily used to load collection items from file with file based collections or where a cache is used.

Creation

The fiftyoneDegreesFilePoolInit method is used to initialise a pointer to a fiftyoneDegreesFilePool. A concurrency value is provided to indicate the maximum number of threads that will be in operation. If this value is lower than the actual number of threads the stack can be exhausted and a null pointer is returned instead of a valid file handle. The concurrency value must always be the same or greater than the number of threads. When compiled in single threaded operation a pool is not strictly required and the implementation maintains a simple stack for consistency of interface and to minimise divergent code.

Get & Release

Handles are retrieved from the pool via the fiftyoneDegreesFileHandleGet method. The handle MUST be returned with the fiftyoneDegreesFileHandleRelease method when it is finished with. The handle will always be open and ready for read only operation. The position of the handle within the source file cannot be assumed. If too many threads are accessing the pool simultaneously, meaning a handle cannot be secured, then a NULL pointer is returned.

Free

The handles are closed when the reader is released via the fiftyoneDegreesFilePoolRelease method. Any memory allocated by the implementation for the stack is freed.

File Operations

Common file operations can also be carried out using the methods defined here. The supported operations are:

copy : fiftyoneDegreesFileCopy

create directory : fiftyoneDegreesFileCreateDirectory

create temp file : fiftyoneDegreesFileCreateTempFile

delete : fiftyoneDegreesFileDelete

get existing temp file : fiftyoneDegreesFileGetExistingTempFile

get file name : fiftyoneDegreesFileGetFileName

get path : fiftyoneDegreesFileGetPath

get size : fiftyoneDegreesFileGetSize

open : fiftyoneDegreesFileOpen

read to byte array : fiftyoneDegreesFileReadToByteArray

write : fiftyoneDegreesFileWrite

Usage Example

FIFTYONE_DEGREES_EXCEPTION_CREATE;

fiftyoneDegreesFilePool pool;

const char *fileName;

// Initialise a file pool

fiftyoneDegreesStatus status = fiftyoneDegreesFilePoolInit(

&filePool,

fileName,

1,

exception);

// Check that the pool was initialised successfully

if (status == FIFTYONE_DEGREES_STATUS_SUCCESS) {

// Get a file handle from the pool

fiftyoneDegreesFileHandle *handle = fiftyoneDegreesFileHandleGet(

&pool,

exception);

// Check that the handle was returned successfully

if (FIFTYONE_DEGREES_EXCEPTION_OKAY) {

// Do something with the file pointer in handle->file

// ...

// Release the file handle back to the pool

fiftyoneDegreesFileHandleRelease(handle);

}

// Release the file pool

fiftyoneDegreesFilePoolRelease(&pool);

}

Design

To improve performance in multi-threaded operation a non locking stack is used where a Compare and Swap (CAS) atomic operation is used to pop and push handles on and off the stack. The design was adapted from the following article (http://nullprogram.com/blog/2014/09/02/) which explains some of the challenges involved including the ABA problem (https://en.wikipedia.org/wiki/ABA_problem). It is for this reason the head structure is implemented as a union between the values and the exchange integer. Pointers are not used as the address space for the stack is continuous and always very small compared to the total addressable memory space.

signed FileOffset – for interop with native API (fseek/ftell)
unsigned FileOffsetUnsigned – as present in data file

if FIFTYONE_DEGREES_LARGE_DATA_FILE_SUPPORT macro is defined, both types will be 64-bit.

Motivation for this is that an offset in a large file beyond 4GB is not addressable by a uint32_t variable.

For files smaller than 4GB uint32_t is fully sufficient to address an offset.

fseek/ftell functions MAY also fail on files larger than 2 GiB due to the limitations of long int type used, so enabling the option MIGHT be necessary in those cases as well.

On POSIX systems _FILE_OFFSET_BITS should be defined as 64 at the same time for off_t (used by fseeko/ftello from <stdio.h>) to be 64-bit as well.

Collaboration diagram for File:

Structs

union	fiftyoneDegreesFileHandle File handle node in the stack of handles. More...
struct	fiftyoneDegreesFilePool Stack of handles used to read data from a single source file. More...

Macros

#define	GET_CURRENT_DIR getcwd Platform specific method pointer to return the current working directory.
#define	TEMP_UNIQUE_STRING_LENGTH 20

#define FIFTYONE_DEGREES_FILE_MAX_PATH 4096

Define the max path length on the target system. More...

Typedefs

typedef uint32_t	fiftyoneDegreesFileOffsetUnsigned Type for collection start offset (in file). More...
typedef long	fiftyoneDegreesFileOffset Type for file offset in API. More...

Functions

int	fiftyoneDegreesFileSeek (FILE *stream, fiftyoneDegreesFileOffset offset, int origin) Moves the file pointer to a specified location. More...
fiftyoneDegreesFileOffset	fiftyoneDegreesFileTell (FILE *stream) Gets the current position of a file pointer. More...
void	fiftyoneDegreesFilePoolRelease (fiftyoneDegreesFilePool *pool) Releases the file handles contained in the pool and frees any internal memory used by the pool. More...
fiftyoneDegreesStatusCode	fiftyoneDegreesFileOpen (const char fileName, FILE *handle) Opens the file path provided placing the file handle in the handle parameter. More...
fiftyoneDegreesStatusCode	fiftyoneDegreesFileWrite (const char fileName, const void data, const size_t length) Writes binary data to the file path provided, closing the file once finished. More...
fiftyoneDegreesStatusCode	fiftyoneDegreesFileCopy (const char source, const char destination) Copy a file from one location to another. More...
fiftyoneDegreesStatusCode	fiftyoneDegreesFileDelete (const char *fileName) Delete a file from the file system. More...
fiftyoneDegreesStatusCode	fiftyoneDegreesFileCreateDirectory (const char *pathName) Creates a directory with the specified path, and returns the result of the operation. More...
fiftyoneDegreesStatusCode	fiftyoneDegreesFileGetPath (const char dataFolderName, const char fileName, char *destination, size_t size) Iterates up the folders from the current working directory until a file in the sub folder dataFolderName with the name fileName is found which can be opened. More...
bool	fiftyoneDegreesFileGetExistingTempFile (const char masterFile, const char paths, int count, fiftyoneDegreesFileOffset bytesToCompare, const char destination) Gets the path to a temporary file which is an exact copy of the master file if one exists. More...
int	fiftyoneDegreesFileDeleteUnusedTempFiles (const char masterFileName, const char *paths, int count, fiftyoneDegreesFileOffset bytesToCompare) Finds all the temporary files which is an exact copy of the master file if any exist. More...
fiftyoneDegreesStatusCode	fiftyoneDegreesFileAddTempFileName (const char masterFileName, char destination, size_t nameStart, size_t length) Create a temporary file name and add it to the destination. More...
fiftyoneDegreesStatusCode	fiftyoneDegreesFileNewTempFile (const char masterFile, const char paths, int count, char destination, size_t length) Create a temporary file containing a copy of the master file using the first writable path in the list of paths provided. More...
DEPRECATED fiftyoneDegreesStatusCode	fiftyoneDegreesFileCreateTempFile (const char masterFile, const char paths, int count, const char destination) [DEPRECATED - Use fiftyoneDegreesFileNewTempFile instead] Create a temporary file containing a copy of the master file using the first writable path in the list of paths provided. More...
fiftyoneDegreesStatusCode	fiftyoneDegreesFilePoolInit (fiftyoneDegreesFilePool filePool, const char fileName, uint16_t concurrency, fiftyoneDegreesException *exception) Initialises the pool with a stack of open read only file handles all associated with the file name. More...
fiftyoneDegreesFileHandle *	fiftyoneDegreesFileHandleGet (fiftyoneDegreesFilePool filePool, fiftyoneDegreesException exception) Retrieves a read only open file handle from the pool. More...
void	fiftyoneDegreesFileHandleRelease (fiftyoneDegreesFileHandle *handle) Returns a handle previously retrieved via fiftyoneDegreesFileHandleGet back to the pool. More...
fiftyoneDegreesFileOffset	fiftyoneDegreesFileGetSize (const char *fileName) Returns the size of a file in bytes, or -1 if the file does not exist or cannot be accessed. More...
fiftyoneDegreesStatusCode	fiftyoneDegreesFileReadToByteArray (const char fileName, fiftyoneDegreesMemoryReader reader) Reads the contents of a file into memory. More...
void	fiftyoneDegreesFilePoolReset (fiftyoneDegreesFilePool *filePool) Resets the pool without releasing any resources. More...
const char *	fiftyoneDegreesFileGetFileName (const char *filePath) Gets the last, file name, segment of the full file path. More...

Macro Definition Documentation

◆ FIFTYONE_DEGREES_FILE_MAX_PATH

#define FIFTYONE_DEGREES_FILE_MAX_PATH 4096

Define the max path length on the target system.

For Windows this is 260 characters, and 4096 on most UNIX systems. The compiler defined values are not used here as although Windows MSVC always defined _MAX_PATH, UNIX systems do not consistently define the PATH_MAX macro resulting in undefined behavior. For this reason the length of 4096 is used as this is almost always the case.

Examples: Hash/GettingStarted.cpp, Hash/MatchMetrics.cpp, Hash/MetaData.cpp, Hash/ReloadFromFile.cpp, Hash/ReloadFromMemory.cpp, and Hash/StronglyTyped.cpp.

Typedef Documentation

◆ fiftyoneDegreesFileOffset

typedef long fiftyoneDegreesFileOffset

Type for file offset in API.

[signed]

◆ fiftyoneDegreesFileOffsetUnsigned

typedef uint32_t fiftyoneDegreesFileOffsetUnsigned

Type for collection start offset (in file).

[unsigned]

Function Documentation

◆ fiftyoneDegreesFileAddTempFileName()

fiftyoneDegreesStatusCode fiftyoneDegreesFileAddTempFileName	(	const char *	masterFileName,
		char *	destination,
		size_t	nameStart,
		size_t	length
	)

Create a temporary file name and add it to the destination.

Parameters: masterFileName - basename of the master file; destination - buffer to write the temporary file name to; nameStart - position of where the name should be added to the destination; length - length available in the buffer

Returns: the result of the file name creation

◆ fiftyoneDegreesFileCopy()

fiftyoneDegreesStatusCode fiftyoneDegreesFileCopy	(	const char *	source,
		const char *	destination
	)

Copy a file from one location to another.

If there was an error while creating or copying the file, then the appropriate status code will be returned. If the status code is anything other than FIFTYONE_DEGREES_STATUS_SUCCESS then the new file will not exist.

Parameters: source - path to the file to copy; destination - path to the file to create

Returns: the result of the copy operation

◆ fiftyoneDegreesFileCreateDirectory()

fiftyoneDegreesStatusCode fiftyoneDegreesFileCreateDirectory

(

const char *

pathName

)

Creates a directory with the specified path, and returns the result of the operation.

intermediate directories will not be created.

Parameters: pathName - path to create

Returns: the result of the operation

◆ fiftyoneDegreesFileCreateTempFile()

DEPRECATED fiftyoneDegreesStatusCode fiftyoneDegreesFileCreateTempFile	(	const char *	masterFile,
		const char **	paths,
		int	count,
		const char *	destination
	)

[DEPRECATED - Use fiftyoneDegreesFileNewTempFile instead] Create a temporary file containing a copy of the master file using the first writable path in the list of paths provided.

The path which is written to (including file name) is written to the destination parameter which must contain enough memory. If no paths are provided, then the working directory used as the destination.

Parameters: masterFile - full path to the file containing the master data to copy; paths - list of paths to use in order of preference; count - number of paths in the array; destination - memory to write the new file path to

Returns: the result of the create operation

◆ fiftyoneDegreesFileDelete()

fiftyoneDegreesStatusCode fiftyoneDegreesFileDelete

(

const char *

fileName

)

Delete a file from the file system.

If there was an error while removing the file, then the appropriate status code will be returned.

Parameters: fileName - path to the file to be deleted

Returns: the result of the delete operation

◆ fiftyoneDegreesFileDeleteUnusedTempFiles()

int fiftyoneDegreesFileDeleteUnusedTempFiles	(	const char *	masterFileName,
		const char **	paths,
		int	count,
		fiftyoneDegreesFileOffset	bytesToCompare
	)

Finds all the temporary files which is an exact copy of the master file if any exist.

If any are found, the method attempts to delete them. The number of successfully deleted temp files is returned. If any files found have the same name as the master file, then false is returned to avoid the same file being treated as if it were a copy by external code. If no paths are provided, then the working directory is searched. Note that this should not be used on Apple systems, as the checks for whether or not a file is in use are not implemented (all files will be deleted regardless of whether they are being used).

Parameters: masterFileName - path to the master file to find a temp version of; paths - list of paths to search in order of preference; count - number of paths in the array; bytesToCompare - number of from the start of the file to compare for equality with the master file, or -1 to compare the whole file

Returns: the number of matching files which have been successfully deleted

◆ fiftyoneDegreesFileGetExistingTempFile()

bool fiftyoneDegreesFileGetExistingTempFile	(	const char *	masterFile,
		const char **	paths,
		int	count,
		fiftyoneDegreesFileOffset	bytesToCompare,
		const char *	destination
	)

Gets the path to a temporary file which is an exact copy of the master file if one exists.

If one is found, true is returned and the path written to the memory pointed to by the destination parameter. If the file found has the same name as the master file, then false is returned to avoid the same file being treated as if it were a copy by external code. If no paths are provided, then the working directory is searched.

Parameters: masterFile - path to the master file to find a temp version of; paths - list of paths to search in order of preference; count - number of paths in the array; bytesToCompare - number of from the start of the file to compare for equality with the master file, or -1 to compare the whole file; destination - memory to write the found file path to

Returns: true if a copy of the master file was found, and its path written to destination

◆ fiftyoneDegreesFileGetFileName()

const char* fiftyoneDegreesFileGetFileName

(

const char *

filePath

)

Gets the last, file name, segment of the full file path.

Parameters: filePath - full path to the file.

Returns: the file name from the file path.

◆ fiftyoneDegreesFileGetPath()

fiftyoneDegreesStatusCode fiftyoneDegreesFileGetPath	(	const char *	dataFolderName,
		const char *	fileName,
		char *	destination,
		size_t	size
	)

Iterates up the folders from the current working directory until a file in the sub folder dataFolderName with the name fileName is found which can be opened.

This is assumed to be the data file required by the caller. Enough memory must be allocated to the destination parameter for the full path to be written. The maximum path length is defined by the macro FIFTYONEDEGREES_FILE_MAX_PATH.

Parameters: dataFolderName - the name of the sub folder which is expected to contain the data file; fileName - the name of the data file; destination - memory to write the absolute path to if the file was found; size - the number of bytes assigned to the destination pointer

Returns: the result of the operation

Examples: Hash/GettingStarted.cpp, Hash/MatchMetrics.cpp, Hash/MetaData.cpp, Hash/ReloadFromFile.cpp, Hash/ReloadFromMemory.cpp, and Hash/StronglyTyped.cpp.

◆ fiftyoneDegreesFileGetSize()

fiftyoneDegreesFileOffset fiftyoneDegreesFileGetSize

(

const char *

fileName

)

Returns the size of a file in bytes, or -1 if the file does not exist or cannot be accessed.

Parameters: fileName - path to the file

Returns: size of file in bytes or -1

◆ fiftyoneDegreesFileHandleGet()

fiftyoneDegreesFileHandle* fiftyoneDegreesFileHandleGet	(	fiftyoneDegreesFilePool *	filePool,
		fiftyoneDegreesException *	exception
	)

Retrieves a read only open file handle from the pool.

The handle retrieve must be returned to the pool using fiftyoneDegreesFileHandleGet and must not be freed or closed directly.

Parameters: filePool - to retrieve the file handle from; exception - pointer to an exception data structure to be used if an exception occurs. See exceptions.h.

Returns: a read only open file handle or NULL if no available handles remain in the pool

◆ fiftyoneDegreesFileHandleRelease()

void fiftyoneDegreesFileHandleRelease

(

fiftyoneDegreesFileHandle *

handle

)

Returns a handle previously retrieved via fiftyoneDegreesFileHandleGet back to the pool.

Parameters: handle - to be returned to the pool

◆ fiftyoneDegreesFileNewTempFile()

fiftyoneDegreesStatusCode fiftyoneDegreesFileNewTempFile	(	const char *	masterFile,
		const char **	paths,
		int	count,
		char *	destination,
		size_t	length
	)

Create a temporary file containing a copy of the master file using the first writable path in the list of paths provided.

The path which is written to (including file name) is written to the destination parameter which must contain enough memory. If no paths are provided, then the working directory used as the destination.

Parameters: masterFile - full path to the file containing the master data to copy; paths - list of paths to use in order of preference; count - number of paths in the array; destination - memory to write the new file path to; length - size of the memory to be written to

Returns: the result of the create operation

◆ fiftyoneDegreesFileOpen()

fiftyoneDegreesStatusCode fiftyoneDegreesFileOpen	(	const char *	fileName,
		FILE **	handle
	)

Opens the file path provided placing the file handle in the handle parameter.

Parameters: fileName - full path to the file to open; handle - to be associated with the open file

Returns: the result of the open operation

◆ fiftyoneDegreesFilePoolInit()

fiftyoneDegreesStatusCode fiftyoneDegreesFilePoolInit	(	fiftyoneDegreesFilePool *	filePool,
		const char *	fileName,
		uint16_t	concurrency,
		fiftyoneDegreesException *	exception
	)

Initialises the pool with a stack of open read only file handles all associated with the file name.

The concurrency parameter determines the number of items in the stack.

Parameters: filePool - to be initialised; fileName - full path to the file to open; concurrency - number of items in the stack; exception - pointer to an exception data structure to be used if an exception occurs. See exceptions.h.

Returns: the result of the open operation

◆ fiftyoneDegreesFilePoolRelease()

void fiftyoneDegreesFilePoolRelease

(

fiftyoneDegreesFilePool *

pool

)

Releases the file handles contained in the pool and frees any internal memory used by the pool.

Does not free the memory pointed to by pool.

Parameters: pool - pointer to the stack of file handles to be release

◆ fiftyoneDegreesFilePoolReset()

void fiftyoneDegreesFilePoolReset

(

fiftyoneDegreesFilePool *

filePool

)

Resets the pool without releasing any resources.

Parameters: filePool - to be reset.

◆ fiftyoneDegreesFileReadToByteArray()

fiftyoneDegreesStatusCode fiftyoneDegreesFileReadToByteArray	(	const char *	fileName,
		fiftyoneDegreesMemoryReader *	reader
	)

Reads the contents of a file into memory.

The correct amount of memory will be allocated by the method. This memory needs to be freed by the caller after the data has been finished with.

Parameters: fileName - path to the source file; reader - to contain the pointer to the memory and the size

Returns: status code indicating whether the read was successful

Examples: Hash/ReloadFromMemory.cpp.

◆ fiftyoneDegreesFileSeek()

int fiftyoneDegreesFileSeek	(	FILE *	stream,
		fiftyoneDegreesFileOffset	offset,
		int	origin
	)

Moves the file pointer to a specified location.

Parameters: stream - Pointer to FILE structure.; offset - Number of bytes from origin.; origin - Initial position.

Returns

◆ fiftyoneDegreesFileTell()

fiftyoneDegreesFileOffset fiftyoneDegreesFileTell

(

FILE *

stream

)

Gets the current position of a file pointer.

Parameters: stream - Target FILE structure.

Returns: The current file position.

◆ fiftyoneDegreesFileWrite()

fiftyoneDegreesStatusCode fiftyoneDegreesFileWrite	(	const char *	fileName,
		const void *	data,
		const size_t	length
	)

Writes binary data to the file path provided, closing the file once finished.

Parameters: fileName - full path to the file to write to; data - binary data to write; length - the number of bytes in the data to be written

Returns: the result of the write operation

51Degrees Device Detection C/C++ 4.5

FileCommon

Detailed Description

Introduction

Creation

Get & Release

Free

File Operations

Usage Example

Design

Collaboration diagram for File:

Structs

Macros

Typedefs

Functions

Macro Definition Documentation

◆ FIFTYONE_DEGREES_FILE_MAX_PATH

Typedef Documentation

◆ fiftyoneDegreesFileOffset

◆ fiftyoneDegreesFileOffsetUnsigned

Function Documentation

◆ fiftyoneDegreesFileAddTempFileName()

◆ fiftyoneDegreesFileCopy()

◆ fiftyoneDegreesFileCreateDirectory()

◆ fiftyoneDegreesFileCreateTempFile()

◆ fiftyoneDegreesFileDelete()

◆ fiftyoneDegreesFileDeleteUnusedTempFiles()

◆ fiftyoneDegreesFileGetExistingTempFile()

◆ fiftyoneDegreesFileGetFileName()

◆ fiftyoneDegreesFileGetPath()

◆ fiftyoneDegreesFileGetSize()

◆ fiftyoneDegreesFileHandleGet()

◆ fiftyoneDegreesFileHandleRelease()

◆ fiftyoneDegreesFileNewTempFile()

◆ fiftyoneDegreesFileOpen()

◆ fiftyoneDegreesFilePoolInit()

◆ fiftyoneDegreesFilePoolRelease()

◆ fiftyoneDegreesFilePoolReset()

◆ fiftyoneDegreesFileReadToByteArray()

◆ fiftyoneDegreesFileSeek()

◆ fiftyoneDegreesFileTell()

◆ fiftyoneDegreesFileWrite()

On This Page

File
Common