Disk ARchive: libdar::parallel_tronconneuse Class Reference

| Disk ARchive 2.8.1 Full featured and portable backup and archiving tool

Loading...
Searching...
No Matches
Public Member Functions | Private Types | Private Member Functions | Static Private Member Functions | Private Attributes | List of all members
libdar::parallel_tronconneuse Class Reference
Private

this is a partial implementation of the generic_file interface to cypher/decypher data block by block. More...

`#include <parallel_tronconneuse.hpp>`

## Public Member Functions
| parallel_tronconneuse (U_I workers, U_32 block_size, generic_file &encrypted_side, const archive_version &reading_ver, std::unique_ptr< crypto_module > &ptr)
| This is the constructor.
| parallel_tronconneuse (const parallel_tronconneuse &ref)=delete
| copy constructor
| parallel_tronconneuse (parallel_tronconneuse &&ref)=default
| move constructor
parallel_tronconneuse & | operator= (const parallel_tronconneuse &ref)=delete
| assignment operator
parallel_tronconneuse & | operator= (parallel_tronconneuse &&ref) noexcept=default
| move operator
| ~parallel_tronconneuse () noexcept
| destructor
virtual bool | skippable (skippability direction, const infinint &amount) override
| inherited from generic_file
virtual bool | skip (const infinint &pos) override
| inherited from generic_file
virtual bool | skip_to_eof () override
| inherited from generic_file
virtual bool | skip_relative (S_I x) override
| inherited from generic_file
virtual bool | truncatable (const infinint &pos) const override
| inherited from generic_file
virtual infinint | get_position () const override
| inherited from generic_file
virtual void | write_end_of_file () override
| in write_only mode indicate that end of file is reached
virtual void | set_initial_shift (const infinint &x) override
| this method to modify the initial shift. This overrides the constructor "no_initial_shift" of the constructor
virtual void | set_callback_trailing_clear_data (trailing_clear_data_callback call_back) override
virtual U_32 | get_clear_block_size () const override
| returns the block size given to constructor

## Private Types
enum class | thread_status { running , suspended , dead }

## Private Member Functions
virtual void | inherited_read_ahead (const infinint &amount) override
| this protected inherited method is now private for inherited classes of tronconneuse
virtual U_I | inherited_read (char *a, U_I size) override
| this protected inherited method is now private for inherited classes of tronconneuse
virtual void | inherited_write (const char *a, U_I size) override
| inherited from generic_file
virtual void | inherited_truncate (const infinint &pos) override
| this prorected inherited method is now private for inherited classed of tronconneuse
virtual void | inherited_sync_write () override
| this protected inherited method is now private for inherited classes of tronconneuse
virtual void | inherited_flush_read () override
| this protected inherited method is now private for inherited classes of tronconneuse
virtual void | inherited_terminate () override
| this protected inherited method is now private for inherited classes of tronconneuse
const archive_version & | get_reading_version () const
bool | send_read_order (tronco_flags order, const infinint &for_offset=0)
| send and order to subthreads and gather acks from them
void | send_write_order (tronco_flags order)
| send order in write mode
void | go_read ()
| wake up threads in read mode when necessary
void | read_refill ()
| fill lus_data/lus_flags from ratelier_gather if these are empty
tronco_flags | purge_ratelier_from_next_order (infinint pos=0)
| purge the ratelier from the next order which is provided as returned value
bool | purge_unack_stop_order (const infinint &pos=0)
| removing the ignore_stop_acks pending on the pipe
bool | find_offset_in_lus_data (const infinint &pos)
| flush lus_data/lus_flags up to requested pos offset to be found or all data has been removed
void | run_threads ()
| reset the interthread datastructure and launch the threads
void | stop_threads ()
| end threads taking into account the fact they may be suspended on the barrier
void | join_workers_only ()
| call by join_threads() below just code simplification around exception handling
void | join_threads ()
| wait for threads to finish and eventually rethrow their exceptions in current thread

## Static Private Member Functions
static U_I | get_ratelier_size (U_I num_worker)
static U_I | get_heap_size (U_I num_worker)

## Private Attributes
U_I | num_workers
| number of worker threads
U_32 | clear_block_size
| size of a clear block
infinint | current_position
| current position for the upper layer perspective (modified by skip*, inherited_read/write, find_offset_in_lus_data)
infinint | initial_shift
| the offset in the "encrypted" below layer at which starts the encrypted data
archive_version | reading_ver
| archive format we follow
std::unique_ptr< crypto_module > | crypto
| the crypto module use to cipher / uncipher block of data
infinint(* | mycallback )(generic_file &below, const archive_version &reading_ver)
generic_file * | encrypted
U_I | ignore_stop_acks
| how much stop ack still to be read (aborted stop order context)
thread_status | t_status
| wehther child thread are waiting us on the barrier
std::deque< std::unique_ptr< crypto_segment > > | lus_data
std::deque< signed int > | lus_flags
bool | lus_eof
bool | check_bytes_to_skip
| whether to check for bytes to skip
std::unique_ptr< crypto_segment > | tempo_write
infinint | block_num
std::shared_ptr< libthreadar::ratelier_scatter< crypto_segment > > | scatter
std::shared_ptr< libthreadar::ratelier_gather< crypto_segment > > | gather
std::shared_ptr< libthreadar::barrier > | waiter
std::shared_ptr< heap< crypto_segment > > | tas
std::deque< std::unique_ptr< crypto_worker > > | travailleur
std::unique_ptr< read_below > | crypto_reader
std::unique_ptr< write_below > | crypto_writer

## Detailed Description

this is a partial implementation of the generic_file interface to cypher/decypher data block by block.

This class is a pure virtual one, as several calls have to be defined by inherited classes

- encrypted_block_size_for
- clear_block_allocated_size_for
- encrypt_data
- decrypt_data

parallel_tronconneuse is either read_only or write_only, read_write is not allowed. The openning mode is defined by encrypted_side's mode. In write_only no skip() is allowed, writing is sequential from the beginning of the file to the end (like writing to a pipe). In read_only all skip() functions are available.

Definition at line 88 of file parallel_tronconneuse.hpp.

## Member Enumeration Documentation

## ◆ thread_status

enum class libdar::parallel_tronconneuse::thread_status
| strongprivate

Definition at line 189 of file parallel_tronconneuse.hpp.

## Constructor & Destructor Documentation

## ◆ parallel_tronconneuse()

libdar::parallel_tronconneuse::parallel_tronconneuse | ( | U_I | workers,
|  | U_32 | block_size,
|  | generic_file & | encrypted_side,
|  | const archive_version & | reading_ver,
|  | std::unique_ptr< crypto_module > & | ptr )

This is the constructor.

Parameters

  [in] | workers | is the number of worker threads
  [in] | block_size | is the size of block encryption (the size of clear data encrypted toghether).
  [in] | encrypted_side | where encrypted data are read from or written to.
  [in] | reading_ver | version of the archive format
  [in] | ptr | pointer to an crypto_module object that will be passed to the parallel_tronconneuse

Note
  that encrypted_side is not owned and destroyed by tronconneuse, it must exist during all the life of the tronconneuse object, and is not destroyed by the tronconneuse's destructor

References reading_ver.

Referenced by operator=(), operator=(), parallel_tronconneuse(), and parallel_tronconneuse().

## Member Function Documentation

## ◆ find_offset_in_lus_data()

bool libdar::parallel_tronconneuse::find_offset_in_lus_data | ( | const infinint & | pos | ) |
| private

flush lus_data/lus_flags up to requested pos offset to be found or all data has been removed

Parameters

  [in] | pos | the data offset we look for

Returns
  true if the offset has been found and is ready for next inherited_read() call, false else and lus_data/lus_flags have been empties of the first non-order blocks found (data blocks)

Note
  current_position is set updated by the amount of of data that has been read/skipped over, from the workers

## ◆ get_clear_block_size()

virtual U_32 libdar::parallel_tronconneuse::get_clear_block_size | ( |  | ) | const
| inlineoverridevirtual

returns the block size given to constructor

Definition at line 153 of file parallel_tronconneuse.hpp.

References clear_block_size.

## ◆ get_position()

virtual infinint libdar::parallel_tronconneuse::get_position | ( |  | ) | const
| inlineoverridevirtual

inherited from generic_file

Implements libdar::generic_file.

Definition at line 132 of file parallel_tronconneuse.hpp.

References current_position, and libdar::generic_file::is_terminated().

## ◆ get_ratelier_size()

U_I libdar::parallel_tronconneuse::get_ratelier_size | ( | U_I | num_worker | ) |
| inlinestaticprivate

Definition at line 308 of file parallel_tronconneuse.hpp.

## ◆ get_reading_version()

const archive_version & libdar::parallel_tronconneuse::get_reading_version | ( |  | ) | const
| inlineprivate

Definition at line 186 of file parallel_tronconneuse.hpp.

## ◆ inherited_flush_read()

virtual void libdar::parallel_tronconneuse::inherited_flush_read | ( |  | ) |
| overrideprivatevirtual

this protected inherited method is now private for inherited classes of tronconneuse

Implements libdar::generic_file.

## ◆ inherited_read()

virtual U_I libdar::parallel_tronconneuse::inherited_read | ( | char * | a,
|  | U_I | size )
| overrideprivatevirtual

this protected inherited method is now private for inherited classes of tronconneuse

Implements libdar::generic_file.

## ◆ inherited_read_ahead()

virtual void libdar::parallel_tronconneuse::inherited_read_ahead | ( | const infinint & | amount | ) |
| overrideprivatevirtual

this protected inherited method is now private for inherited classes of tronconneuse

Implements libdar::generic_file.

## ◆ inherited_sync_write()

virtual void libdar::parallel_tronconneuse::inherited_sync_write | ( |  | ) |
| overrideprivatevirtual

this protected inherited method is now private for inherited classes of tronconneuse

Implements libdar::generic_file.

## ◆ inherited_terminate()

virtual void libdar::parallel_tronconneuse::inherited_terminate | ( |  | ) |
| overrideprivatevirtual

this protected inherited method is now private for inherited classes of tronconneuse

Implements libdar::generic_file.

References reading_ver.

## ◆ inherited_truncate()

virtual void libdar::parallel_tronconneuse::inherited_truncate | ( | const infinint & | pos | ) |
| inlineoverrideprivatevirtual

this prorected inherited method is now private for inherited classed of tronconneuse

Note
  no skippability in write mode, so no truncate possibility neither, this call will always throw a bug exception

Implements libdar::generic_file.

Definition at line 174 of file parallel_tronconneuse.hpp.

## ◆ inherited_write()

virtual void libdar::parallel_tronconneuse::inherited_write | ( | const char * | a,
|  | U_I | size )
| overrideprivatevirtual

inherited from generic_file

this protected inherited method is now private for inherited classes of tronconneuse

Implements libdar::generic_file.

## ◆ purge_ratelier_from_next_order()

tronco_flags libdar::parallel_tronconneuse::purge_ratelier_from_next_order | ( | infinint | pos = 0 | ) |
| private

purge the ratelier from the next order which is provided as returned value

Parameters

  [in] | pos | if pos is not zero, the normal data located at pos offset is also looked for. If it is found before any order the call returns tronco_flags::normal and the order is not purged but the ignore_stop_acks fields is set to true for further reading to skip over this order acknolegments.

## ◆ purge_unack_stop_order()

bool libdar::parallel_tronconneuse::purge_unack_stop_order | ( | const infinint & | pos = 0 | ) |
| private

removing the ignore_stop_acks pending on the pipe

Parameters

  [in] | pos | if pos is not zero and normal data is found at pos offset before all stop acks could get read, the call stops and return false. Else true is returned meaning all stop acks has been read and removed from the pipe

Note
  this method acts the same pas purge_ratelier_from_next_order() only if a pending order was tried to be purged by this call but aborted because the data at the expected offset has been found before. If data at given offset is found in local thread data buffer or if no order is pending for ack, the call does not try purging anothing from the pipe. Else, it completes the purging process (for stop orders) as started by purge_ratelier_from_next_order()

## ◆ send_read_order()

bool libdar::parallel_tronconneuse::send_read_order | ( | tronco_flags | order,
|  | const infinint & | for_offset = 0 )
| private

send and order to subthreads and gather acks from them

Parameters

  [in] | order | is the order to send to the subthreads
  [in] | for_offset | is not zero is the offset we want to skip to (it is only taken into account when order is stop)

Note
  with order stop and non zero for_offset, if the data at for_offset is found while purging the ratelier_gather, the purge is stopped, false is returned and the ignore_stop field is set to true, meaning that a stop order has not been purged from the pipe and should be ignored when its acknolegements will be met. In any other case true is returned, the subthreaded got the order and the ratelier has been purged.

## ◆ set_callback_trailing_clear_data()

virtual void libdar::parallel_tronconneuse::set_callback_trailing_clear_data | ( | trailing_clear_data_callback | call_back | ) |
| overridevirtual

let the caller give a callback function that given a generic_file with cyphered data, is able to return the offset of the first clear byte located after all the cyphered data, this callback function is used (if defined by the following method), when reaching End of File.

## ◆ skip()

virtual bool libdar::parallel_tronconneuse::skip | ( | const infinint & | pos | ) |
| overridevirtual

inherited from generic_file

Implements libdar::generic_file.

References skip().

Referenced by skip().

## ◆ skip_relative()

virtual bool libdar::parallel_tronconneuse::skip_relative | ( | S_I | x | ) |
| overridevirtual

inherited from generic_file

Implements libdar::generic_file.

References skip_relative().

Referenced by skip_relative().

## ◆ skip_to_eof()

virtual bool libdar::parallel_tronconneuse::skip_to_eof | ( |  | ) |
| overridevirtual

inherited from generic_file

Implements libdar::generic_file.

References skip_to_eof().

Referenced by skip_to_eof().

## ◆ skippable()

virtual bool libdar::parallel_tronconneuse::skippable | ( | skippability | direction,
|  | const infinint & | amount )
| overridevirtual

inherited from generic_file

Implements libdar::generic_file.

References skippable().

Referenced by skippable().

## ◆ truncatable()

virtual bool libdar::parallel_tronconneuse::truncatable | ( | const infinint & | pos | ) | const
| inlineoverridevirtual

inherited from generic_file

Implements libdar::generic_file.

Definition at line 130 of file parallel_tronconneuse.hpp.

References truncatable().

Referenced by truncatable().

## ◆ write_end_of_file()

virtual void libdar::parallel_tronconneuse::write_end_of_file | ( |  | ) |
| inlineoverridevirtual

in write_only mode indicate that end of file is reached

this call must be called in write mode to purge the internal cache before deleting the object (else some data may be lost) no further write call is allowed

Note
  this call cannot be used from the destructor, because it relies on pure virtual methods

Definition at line 140 of file parallel_tronconneuse.hpp.

References libdar::generic_file::is_terminated(), and libdar::generic_file::sync_write().

## Member Data Documentation

## ◆ block_num

infinint libdar::parallel_tronconneuse::block_num
| private

Definition at line 220 of file parallel_tronconneuse.hpp.

## ◆ check_bytes_to_skip

bool libdar::parallel_tronconneuse::check_bytes_to_skip
| private

whether to check for bytes to skip

Definition at line 215 of file parallel_tronconneuse.hpp.

## ◆ clear_block_size

U_32 libdar::parallel_tronconneuse::clear_block_size
| private

size of a clear block

Definition at line 194 of file parallel_tronconneuse.hpp.

Referenced by get_clear_block_size().

## ◆ crypto

std::unique_ptr<crypto_module> libdar::parallel_tronconneuse::crypto
| private

the crypto module use to cipher / uncipher block of data

Definition at line 198 of file parallel_tronconneuse.hpp.

## ◆ crypto_reader

std::unique_ptr<read_below> libdar::parallel_tronconneuse::crypto_reader
| private

Definition at line 232 of file parallel_tronconneuse.hpp.

## ◆ crypto_writer

std::unique_ptr<write_below> libdar::parallel_tronconneuse::crypto_writer
| private

Definition at line 233 of file parallel_tronconneuse.hpp.

## ◆ current_position

infinint libdar::parallel_tronconneuse::current_position
| private

current position for the upper layer perspective (modified by skip*, inherited_read/write, find_offset_in_lus_data)

Definition at line 195 of file parallel_tronconneuse.hpp.

Referenced by get_position().

## ◆ encrypted

generic_file* libdar::parallel_tronconneuse::encrypted
| private

Definition at line 200 of file parallel_tronconneuse.hpp.

## ◆ gather

std::shared_ptr<libthreadar::ratelier_gather<crypto_segment> > libdar::parallel_tronconneuse::gather
| private

Definition at line 225 of file parallel_tronconneuse.hpp.

## ◆ ignore_stop_acks

U_I libdar::parallel_tronconneuse::ignore_stop_acks
| private

how much stop ack still to be read (aborted stop order context)

Definition at line 204 of file parallel_tronconneuse.hpp.

## ◆ initial_shift

infinint libdar::parallel_tronconneuse::initial_shift
| private

the offset in the "encrypted" below layer at which starts the encrypted data

Definition at line 196 of file parallel_tronconneuse.hpp.

## ◆ lus_data

std::deque<std::unique_ptr<crypto_segment> > libdar::parallel_tronconneuse::lus_data
| private

Definition at line 212 of file parallel_tronconneuse.hpp.

## ◆ lus_eof

bool libdar::parallel_tronconneuse::lus_eof
| private

Definition at line 214 of file parallel_tronconneuse.hpp.

## ◆ lus_flags

std::deque<signed int> libdar::parallel_tronconneuse::lus_flags
| private

Definition at line 213 of file parallel_tronconneuse.hpp.

## ◆ mycallback

infinint(* libdar::parallel_tronconneuse::mycallback) (generic_file &below, const archive_version &reading_ver)
| private

Definition at line 199 of file parallel_tronconneuse.hpp.

## ◆ num_workers

U_I libdar::parallel_tronconneuse::num_workers
| private

number of worker threads

Definition at line 193 of file parallel_tronconneuse.hpp.

## ◆ reading_ver

archive_version libdar::parallel_tronconneuse::reading_ver
| private

archive format we follow

Definition at line 197 of file parallel_tronconneuse.hpp.

Referenced by inherited_terminate(), and parallel_tronconneuse().

## ◆ scatter

std::shared_ptr<libthreadar::ratelier_scatter<crypto_segment> > libdar::parallel_tronconneuse::scatter
| private

Definition at line 224 of file parallel_tronconneuse.hpp.

## ◆ t_status

thread_status libdar::parallel_tronconneuse::t_status
| private

wehther child thread are waiting us on the barrier

Definition at line 205 of file parallel_tronconneuse.hpp.

## ◆ tas

std::shared_ptr<heap<crypto_segment> > libdar::parallel_tronconneuse::tas
| private

Definition at line 227 of file parallel_tronconneuse.hpp.

## ◆ tempo_write

std::unique_ptr<crypto_segment> libdar::parallel_tronconneuse::tempo_write
| private

Definition at line 219 of file parallel_tronconneuse.hpp.

## ◆ travailleur

std::deque<std::unique_ptr<crypto_worker> > libdar::parallel_tronconneuse::travailleur
| private

Definition at line 231 of file parallel_tronconneuse.hpp.

## ◆ waiter

std::shared_ptr<libthreadar::barrier> libdar::parallel_tronconneuse::waiter
| private

Definition at line 226 of file parallel_tronconneuse.hpp.

The documentation for this class was generated from the following file:

- parallel_tronconneuse.hpp