ESDM
Middleware for Earth System Data
Data Structures | Macros | Functions
esdm-scheduler.c File Reference

The scheduler receives application requests and schedules subsequent I/O requests as are necessary for metadata lookups and data reconstructions. More...

#include <esdm-internal.h>
#include <esdm.h>
#include <glib.h>
#include <math.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
Include dependency graph for esdm-scheduler.c:

Data Structures

struct  dimInfo_t
 

Macros

#define _GNU_SOURCE
 
#define DEBUG_ENTER   ESDM_DEBUG_COM_FMT("SCHEDULER", "", "")
 
#define DEBUG(fmt, ...)   ESDM_DEBUG_COM_FMT("SCHEDULER", fmt, __VA_ARGS__)
 

Functions

esdm_scheduler_tesdm_scheduler_init (esdm_instance_t *esdm)
 
esdm_status esdm_scheduler_finalize (esdm_instance_t *esdm)
 
double esdmI_backendOutputTime ()
 
double esdmI_backendInputTime ()
 
void esdmI_resetBackendIoTimes ()
 
esdm_copyTimes_t esdmI_performance_copy ()
 
esdm_status esdm_dataspace_copy_data (esdm_dataspace_t *sourceSpace, void *voidPtrSource, esdm_dataspace_t *destSpace, void *voidPtrDest)
 
bool esdmI_scheduler_try_direct_io (esdm_fragment_t *f, void *buf, esdm_dataspace_t *da)
 
esdm_status esdm_scheduler_enqueue_read (esdm_instance_t *esdm, io_request_status_t *status, int frag_count, esdm_fragment_t **read_frag, void *buf, esdm_dataspace_t *buf_space)
 
esdm_status esdmI_scheduler_readSingleFragmentBlocking (esdm_instance_t *esdm, esdm_dataset_t *dataset, void *buffer, esdm_dataspace_t *memspace, esdm_fragment_t *fragment)
 
esdmI_hypercubeSet_tesdm_scheduler_makeSplitRecommendation (esdm_dataspace_t *space, esdm_backend_t *backend)
 
esdm_status esdm_scheduler_enqueue_write (esdm_instance_t *esdm, io_request_status_t *status, esdm_dataset_t *dataset, void *buf, esdm_dataspace_t *space, bool requestIsInternal)
 
void esdmI_scheduler_writeFragmentNonblocking (esdm_instance_t *esdm, esdm_fragment_t *fragment, bool requestIsInternal, io_request_status_t *status)
 
esdm_status esdmI_scheduler_writeFragmentBlocking (esdm_instance_t *esdm, esdm_fragment_t *fragment, bool requestIsInternal)
 
esdm_status esdm_scheduler_status_init (io_request_status_t *status)
 
esdm_status esdm_scheduler_status_finalize (io_request_status_t *status)
 
esdm_status esdm_scheduler_wait (io_request_status_t *status)
 
esdm_status esdm_scheduler_write_blocking (esdm_instance_t *esdm, esdm_dataset_t *dataset, void *buf, esdm_dataspace_t *subspace, bool requestIsInternal)
 
esdm_status esdm_scheduler_read_blocking (esdm_instance_t *esdm, esdm_dataset_t *dataset, void *buf, esdm_dataspace_t *subspace, esdmI_hypercubeSet_t **out_fillRegion, bool allowWriteback, bool requestIsInternal)
 
esdm_readTimes_t esdmI_performance_read ()
 
esdm_writeTimes_t esdmI_performance_write ()
 

Detailed Description

The scheduler receives application requests and schedules subsequent I/O requests as are necessary for metadata lookups and data reconstructions.

Function Documentation

◆ esdm_dataspace_copy_data()

esdm_status esdm_dataspace_copy_data ( esdm_dataspace_t sourceSpace,
void *  sourceData,
esdm_dataspace_t destSpace,
void *  destData 
)

Copy data from one buffer to another, possibly partially, possibly rearranging the data as prescribed by the given dataspaces.

This function copies all the data that is contained within the intersection of the two dataspaces from the source buffer to the destination buffer. The order and layout of the data elements in each buffer is described by the associated dataspace, allowing this function to be used to

  • pack non-contiguous (= strided) data into a contiguous buffer
  • unpack a contiguous buffer into a larger, non-contiguous (= strided) dataspace
  • transpose data (for example from FORTRAN order to C order and vice versa)
  • reverse the order of the data in one or more dimensions (rather esoteric use!)

In all cases, only the intersection of the two hypercubes described by the two dataspaces is copied: If the source space is larger, only the overlapping part will be read, and if the destination space is larger, only the overlapping part will be written to. If the two dataspaces don't intersect, nothing will be done.

Parameters
[in]sourceSpacedataspace that describes the layout of the sourceData buffer
[in]sourceDatapointer to the first source data element, the logical coordinate of this data element is the offset of the source dataspace
[in]destSpacedataspace that describes the layout of the destData buffer
[out]destDatapointer to the first destination data element, the logical coordinate of this data element is the offset of the destination dataspace
Returns
status

◆ esdm_scheduler_read_blocking()

esdm_status esdm_scheduler_read_blocking ( esdm_instance_t esdm,
esdm_dataset_t dataset,
void *  buf,
esdm_dataspace_t memspace,
esdmI_hypercubeSet_t **  out_fillRegion,
bool  allowWriteback,
bool  requestIsInternal 
)

Calls to reads have to be completed before they can return to the application and are therefor blocking.

Note: write is also blocking right now.

Parameters
[out]out_fillRegionReturns a pointer to a hypercube set that covers the region for which no data was found. It's the callers' responsibility to either pass NULL or to destroy the hypercube set themselves.