SDCard

SDCard-based miniscopes, also known as the “wire free” miniscopes

class mio.devices.sdcard.SDBufferHeader(*, linked_list: int, frame_num: int, buffer_count: int, frame_buffer_count: int, write_buffer_count: int, dropped_buffer_count: int, timestamp: int, write_timestamp: int | None = None, length: int, data_length: int, battery_voltage: int | None = None)

Header data at the start of each frame

POSITIONS: ClassVar[dict[str, int]] = {'buffer_count': 3, 'data_length': 8, 'dropped_buffer_count': 6, 'frame_buffer_count': 4, 'frame_num': 2, 'length': 0, 'linked_list': 1, 'timestamp': 7, 'write_buffer_count': 5}
battery_voltage: int | None
data_length: int
dropped_buffer_count: int
length: int
model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

write_buffer_count: int
write_timestamp: int | None
class mio.devices.sdcard.SDCardDevice(drive: str | Path, layout: SDLayout | Path | PathLike[str] | Annotated[str, FieldInfo(annotation=NoneType, required=True, metadata=[_PydanticGeneralMetadata(pattern='[\\w\\-\\/#]+')])] = 'wirefree-sd-layout')

I/O for data on an SDCardDevice

an instance of sdcard.SDLayout (typically in formats ) configures how the data is laid out on the SD card. This class makes the i/o operations abstract over multiple layouts

Parameters:
check_valid() → bool

Checks that the header sector has the appropriate write keys in it

Returns:

bool - True if valid, False if not

property config: SDConfig

Read configuration from SD Card

property frame: int | None

When reading, the number of the frame that would be read if we were to call read()

property frame_count: int

Total number of frames in recording.

Inferred from n_buffers_recorded and reading a single frame to get the number of buffers per frame.

property position: int | None

When entered as context manager, the current position of the internal file descriptor

positions

A mapping between frame number and byte position in the video that makes for faster seeking :)

As we read, we store the locations of each frame before reading it. Later, we can assign to frame to seek back to those positions. Assigning to frame works without caching position, but has to manually iterate through each frame.

read(return_header: Literal[True] = True) → SDCardFrame
read(return_header: Literal[False] = False) → ndarray

Read a single frame

Parameters:

return_header (bool) – If True, return headers from individual buffers (default False)

Returns:

numpy.ndarray , or a tuple(ndarray, List[SDBufferHeader]) if return_header is True

skip() → None

Skip a frame

Read the buffer headers to determine buffer sizes and just seek ahead

to_img(path: Path | str | None, frame: int | None = None, force: bool = False, chunk_size: int = 1000000.0, progress: bool = True) → None

Create a new disk image that is truncated to the actual size of the video data

Typically, making disk images using dd or other tools will create an image file that is the full size of the media it’s stored on. Rather than sending a bunch of 30GB image files around, we can instead create an image that is truncated to just the size of the data that has actually been recorded

Parameters:

  • path (pathlib.Path) – Path to write .img file to

  • frame (int) – Optional, if present only write the first n frames. If None , write all frames

  • force (bool) – If True, overwrite an existing file. If False , (default) don’t.

  • chunk_size (int) – Number of bytes to read/write at once (default 1e6 )

  • progress (bool) – If True (default), show progress bar

to_video(path: Path | str, fourcc: Literal['GREY', 'mp4v', 'XVID'] = 'GREY', isColor: bool = False, force: bool = False, progress: bool = True) → None

Save contents of SD card to video with opencv

Parameters:

  • path (pathlib.Path) – Output video path, with video extension .avi or .mp4

  • fourcc (str) –

    FourCC code used with opencv. Other codecs may be available depending on your opencv installation, but by default opencv supports one of:

    • GREY (default)

    • mp4v

    • XVID

  • isColor (bool) – Indicates whether output video is in color (default: False)

  • force (bool) – If True, overwrite output video if one already exists (default: False)

  • progress (bool) – If True (default) show progress bar.

class mio.devices.sdcard.SDCardFrame(*, frame: NDArray[Any, Any], headers: list[SDBufferHeader])

An individual frame from a miniscope recording

Typically returned from SDCardDevice.read()

frame: NDArray
property frame_num: int | None

Frame number for this set of headers, if headers are present

classmethod frame_nums_must_be_equal(v: list[SDBufferHeader]) → list[SDBufferHeader] | None

Each frame_number field in each header must be the same (they come from the same frame!)

headers: list[SDBufferHeader]
model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class mio.devices.sdcard.SDCardVideo(*, frames: list[SDCardFrame])

A collection of frames from a miniscope recording

flatten_headers(as_dict: Literal[False]) → list[SDBufferHeader]
flatten_headers(as_dict: Literal[True]) → list[dict]

Return flat list of headers, not grouped by frame

Parameters:

as_dict (bool) – If True, return a list of dictionaries, if False (default), return a list of SDBufferHeader s.

frames: list[SDCardFrame]
model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

to_df(what: Literal['headers'] = 'headers') → DataFrame

Convert frames to pandas dataframe

Parameters:

what ('headers') – What information from the frame to include in the df, currently only ‘headers’ is possible

class mio.devices.sdcard.SDConfig(*, width: int, height: int, fs: int, buffer_size: int, n_buffers_recorded: int, n_buffers_dropped: int)

The configuration of a recording taken on this SD card.

Read from the locations given in ConfigPositions for an SD card with a given SDLayout

buffer_size: int
fs: int
height: int
model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

n_buffers_dropped: int
n_buffers_recorded: int
width: int
class mio.devices.sdcard.SDHeaderPositions(*, gain: int = 4, led: int = 5, ewl: int = 6, record_length: int = 7, fs: int = 8, delay_start: int | None = None, battery_cutoff: int | None = None)

Positions in the header for the whole SD card

battery_cutoff: int | None
delay_start: int | None
ewl: int
fs: int

Frame rate

gain: int
led: int
model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

record_length: int
class mio.devices.sdcard.SDLayout(*, id: Annotated[str, _PydanticGeneralMetadata(pattern='[\\w\\-\\/#]+')], mio_model: Annotated[str, AfterValidator(func=_is_identifier)] = None, mio_version: str = '0.11.1.dev47+gd33cd3a', sectors: SectorConfig, write_key0: int = 226277911, write_key1: int = 226277911, write_key2: int = 226277911, write_key3: int = 226277911, word_size: int = 4, header: SDHeaderPositions = SDHeaderPositions(gain=4, led=5, ewl=6, record_length=7, fs=8, delay_start=None, battery_cutoff=None), config: ConfigPositions = ConfigPositions(width=0, height=1, fs=2, buffer_size=3, n_buffers_recorded=4, n_buffers_dropped=5))

Data layout of an SD Card.

Used by the io.SDCardDevice class to tell it how data on the SD card is laid out.

config: ConfigPositions
header: SDHeaderPositions
model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

sectors: SectorConfig
word_size: int

I’m actually not sure what this is, but 4 is hardcoded a few times in the existing notebook and it appears to be used as a word size when reading from the SD card.

write_key0: int
write_key1: int
write_key2: int
write_key3: int

These don’t seem to actually be used in the existing reading/writing code, but we will leave them here for continuity’s sake :)