Skip to content

cleaner separation of posix client code and library api #651

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 26, 2021

Conversation

MichaelBrim
Copy link
Collaborator

Description

This PR includes code cleanup and restructuring of the client libraries to make sure the library API (i.e., libunifyfs_api) does not rely on any of the POSIX I/O wrapping code. The support for file transfers in the library API has also been updated to use a new server-based transfer scheme where servers are responsible for writing their local data to the destination file. A summary of the various cleanup/changes follows:

client changes:

  • remove unused app_id parameter to unifyfs_mount()
  • Makefile updates to clearly indicate source files corresponding to
    core functionality, library API, and POSIX I/O support
  • adds unifyfs_client state structure to avoid use of global variables
  • move implementation of client API functions to client_api.c
  • move wrapper macros to unifyfs_wrap.h
  • introduces posix_client.[ch] to manage global state for POSIX I/O
  • add unifyfs_fid.[ch] to contain all unifyfs_fid_xxx() functions
  • PMPI wrapper for unifyfs_mount() now checks for and uses environment
    variable setting for UNIFYFS_MOUNTPOINT
  • library API transfer support now uses server-based transfers
  • adds checks in library API to ensure created/accessed files are within
    client namespace

common changes:

  • arraylist_add() now returns position of appended item
  • remove MDHIM-specific configuration settings
  • remove unused constants from unifyfs_const.h

server changes:

  • implements support for parallel transfers where each server with
    local file data copies it to the destination file

tests/examples changes:

  • use updated client and library APIs
  • new library API transfer test
  • add library API support to examples testutil.h
  • new write-transfer example
  • fix sharness.sh test_dir_is_empty() to ignore trailing slashes

How Has This Been Tested?

Tested in Ubuntu Docker environment and on OLCF Summit

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Performance enhancement (non-breaking change which improves efficiency)
  • Code cleanup (non-breaking change which makes code smaller or more readable)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Testing (addition of new tests or update to current tests)
  • Documentation (a change to man pages or other documentation)

Checklist:

  • My code follows the UnifyFS code style requirements.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.
  • All commit messages are properly formatted.

@MichaelBrim MichaelBrim force-pushed the client-api-separation branch from cb89f9e to 7add62e Compare July 21, 2021 18:30
@MichaelBrim MichaelBrim removed the WIP label Jul 21, 2021
@MichaelBrim MichaelBrim force-pushed the client-api-separation branch from 7add62e to 5dacca5 Compare July 21, 2021 18:38
client changes:
* remove unused app_id parameter to unifyfs_mount()
* Makefile updates to clearly indicate source files corresponding to
  core functionality, library API, and POSIX I/O support
* adds unifyfs_client state structure to avoid use of global variables
* move implementation of client API functions to client_api.c
* move wrapper macros to unifyfs_wrap.h
* introduces posix_client.[ch] to manage global state for POSIX I/O
* add unifyfs_fid.[ch] to contain all 'unifyfs_fid_xxx' functions
* PMPI wrapper for unifyfs_mount() now checks for and uses environment
  variable setting for UNIFYFS_MOUNTPOINT
* library API transfer support now uses server-based transfers
* adds checks in library API to ensure created/accessed files are within
  client namespace

common changes:
* arraylist_add() now returns position of appended item
* remove MDHIM-specific configuration settings
* remove unused constants from unifyfs_const.h

server changes:
* implements support for parallel transfers where each server with
  local file data copies it to the destination file

tests/examples changes:
* use updated client and library APIs
* new library API transfer test
* add library API support to examples testutil
* new write-transfer example
* fix sharness.sh test_dir_is_empty() to ignore trailing slashes

TEST_CHECKPATCH_SKIP_FILES="common/src/unifyfs_configurator.h"
TEST_CHECKPATCH_SKIP_FILES+=",client/src/unifyfs_wrap.h"
TEST_CHECKPATCH_SKIP_FILES+=",examples/src/testutil.h"
@MichaelBrim MichaelBrim force-pushed the client-api-separation branch from 5dacca5 to 8cc7eac Compare July 21, 2021 18:54
@MichaelBrim MichaelBrim marked this pull request as ready for review July 21, 2021 20:50
@adammoody adammoody merged commit 3da95de into LLNL:dev Jul 26, 2021
@MichaelBrim MichaelBrim deleted the client-api-separation branch July 29, 2021 20:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants