{"name":"napari-deeplabcut","display_name":"napari DeepLabCut","visibility":"public","icon":"","categories":[],"schema_version":"0.2.0","on_activate":null,"on_deactivate":null,"contributions":{"commands":[{"id":"napari-deeplabcut.get_hdf_reader","title":"Open data with napari DeepLabCut","python_name":"napari_deeplabcut._reader:get_hdf_reader","short_title":null,"category":null,"icon":null,"enablement":null},{"id":"napari-deeplabcut.get_image_reader","title":"Open images with napari DeepLabCut","python_name":"napari_deeplabcut._reader:get_image_reader","short_title":null,"category":null,"icon":null,"enablement":null},{"id":"napari-deeplabcut.get_video_reader","title":"Open videos with napari DeepLabCut","python_name":"napari_deeplabcut._reader:get_video_reader","short_title":null,"category":null,"icon":null,"enablement":null},{"id":"napari-deeplabcut.get_folder_parser","title":"Open folder with napari DeepLabCut","python_name":"napari_deeplabcut._reader:get_folder_parser","short_title":null,"category":null,"icon":null,"enablement":null},{"id":"napari-deeplabcut.get_config_reader","title":"Open config with napari DeepLabCut","python_name":"napari_deeplabcut._reader:get_config_reader","short_title":null,"category":null,"icon":null,"enablement":null},{"id":"napari-deeplabcut.write_hdf","title":"Save keypoint annotations with napari DeepLabCut","python_name":"napari_deeplabcut._writer:write_hdf","short_title":null,"category":null,"icon":null,"enablement":null},{"id":"napari-deeplabcut.write_masks","title":"Save segmentation masks with napari DeepLabCut","python_name":"napari_deeplabcut._writer:write_masks","short_title":null,"category":null,"icon":null,"enablement":null},{"id":"napari-deeplabcut.make_keypoint_controls","title":"Make keypoint controls","python_name":"napari_deeplabcut._widgets:KeypointControls","short_title":null,"category":null,"icon":null,"enablement":null}],"readers":[{"command":"napari-deeplabcut.get_hdf_reader","filename_patterns":["*.h5"],"accepts_directories":false},{"command":"napari-deeplabcut.get_image_reader","filename_patterns":["*.png","*.jpg","*.jpeg"],"accepts_directories":true},{"command":"napari-deeplabcut.get_video_reader","filename_patterns":["*.mp4","*.mov","*.avi"],"accepts_directories":true},{"command":"napari-deeplabcut.get_config_reader","filename_patterns":["*.yaml"],"accepts_directories":false},{"command":"napari-deeplabcut.get_folder_parser","filename_patterns":["*"],"accepts_directories":true}],"writers":[{"command":"napari-deeplabcut.write_hdf","layer_types":["points{1}"],"filename_extensions":[".h5"],"display_name":""},{"command":"napari-deeplabcut.write_masks","layer_types":["shapes{1}"],"filename_extensions":[".csv"],"display_name":""}],"widgets":[{"command":"napari-deeplabcut.make_keypoint_controls","display_name":"Keypoint controls","autogenerate":false}],"sample_data":null,"themes":null,"menus":{},"submenus":null,"keybindings":null,"configuration":[]},"package_metadata":{"metadata_version":"2.1","name":"napari-deeplabcut","version":"0.2.1.2","dynamic":null,"platform":null,"supported_platform":null,"summary":"napari + DeepLabCut annotation tool","description":"# napari-deeplabcut: keypoint annotation for pose estimation\n\n\n\n![\"napari+deeplabcut\"](\"https://images.squarespace-cdn.com/content/v1/57f6d51c9f74566f55ecf271/1d409ffe-c9f4-47e1-bde2-3010c1c40455/naparidlc.png?format=750w\" "\\"napari-deeplabcut\\"")
\n\n[📚Documentation](https://deeplabcut.github.io/DeepLabCut/README.html) |\n[🛠️ DeepLabCut Installation](https://deeplabcut.github.io/DeepLabCut/docs/installation.html) |\n[🌎 Home Page](https://www.deeplabcut.org) |\n\n[](https://www.gnu.org/licenses/bsd3)\n[](https://pypi.org/project/napari-deeplabcut)\n[](https://python.org)\n[](https://github.com/DeepLabCut/napari-deeplabcut/actions)\n[](https://codecov.io/gh/DeepLabCut/napari-deeplabcut)\n[](https://napari-hub.org/plugins/napari-deeplabcut)\n\nA napari plugin for keypoint annotation, also used within DeepLabCut!\n\n\n## Installation\n\nIf you installed DeepLabCut[gui], this plugin is already installed. However, you can also use this as a stand-alone keypoint annotator without using DeepLabCut. Instructions below!\n\nStart by installing PySide6 with `pip install \"pyside6<6.3.2\"`; this is the library we now use to build GUIs.\n\nYou can then install `napari-deeplabcut` via [pip]:\n\n pip install napari-deeplabcut\n\n\n\nAlternatively, to install the latest development version, run:\n\n pip install git+https://github.com/DeepLabCut/napari-deeplabcut.git\n\n\n## Usage\n\nTo use the plugin, please run:\n\n napari\n\nThen, activate the plugin in Plugins > napari-deeplabcut: Keypoint controls.\n\nAll accepted files (config.yaml, images, h5 data files) can be loaded\neither by dropping them directly onto the canvas or via the File menu.\n\nThe easiest way to get started is to drop a folder (typically a folder from within a DeepLabCut's `labeled-data` directory), and, if labeling from scratch, drop the corresponding `config.yaml` to automatically add a `Points layer` and populate the dropdown menus.\n\n[🎥 DEMO\n](https://youtu.be/hsA9IB5r73E)\n\n**Tools & shortcuts are:**\n\n- `2` and `3`, to easily switch between labeling and selection mode\n- `4`, to enable pan & zoom (which is achieved using the mouse wheel or finger scrolling on the Trackpad)\n- `M`, to cycle through regular (sequential), quick, and cycle annotation mode (see the description [here](https://github.com/DeepLabCut/napari-deeplabcut/blob/5a5709dd38868341568d66eab548ae8abf37cd63/src/napari_deeplabcut/keypoints.py#L25-L34))\n- `E`, to enable edge coloring (by default, if using this in refinement GUI mode, points with a confidence lower than 0.6 are marked\nin red)\n- `F`, to toggle between animal and body part color scheme.\n- `V`, to toggle visibility of the selected layer.\n- `backspace` to delete a point.\n- Check the box \"display text\" to show the label names on the canvas.\n- To move to another folder, be sure to save (Ctrl+S), then delete the layers, and re-drag/drop the next folder.\n\n\n### Save Layers\n\nAnnotations and segmentations are saved with `File > Save Selected Layer(s)...` (or its shortcut `Ctrl+S`).\nOnly when saving segmentation masks does a save file dialog pop up to name the destination folder;\nkeypoint annotations are otherwise automatically saved in the corresponding folder as `CollectedData_.h5`.\n- As a reminder, DLC will only use the H5 file; so be sure if you open already labeled images you save/overwrite the H5.\n- Note, before saving a layer, make sure the points layer is selected. If the user clicked on the image(s) layer first, does `Save As`, then closes the window, any labeling work during that session will be lost!\n- Modifying and then saving points in a `machinelabels...` layer will add to or overwrite the existing `CollectedData` layer and will **not** save to the `machinelabels` file.\n\n\n### Video frame extraction and prediction refinement\n\nSince v0.0.4, videos can be viewed in the GUI.\n\nSince v0.0.5, trailing points can be visualized; e.g., helping in the identification\nof swaps or outlier, jittery predictions.\n\nLoading a video (and its corresponding output h5 file) will enable the video actions\nat the top of the dock widget: they offer the option to manually extract video\nframes from the GUI, or to define cropping coordinates.\nNote that keypoints can be displaced and saved, as when annotating individual frames.\n\n\n## Workflow\n\nSuggested workflows, depending on the image folder contents:\n\n1. **Labeling from scratch** – the image folder does not contain `CollectedData_.h5` file.\n\n Open *napari* as described in [Usage](#usage) and open an image folder together with the DeepLabCut project's `config.yaml`.\n The image folder creates an *image layer* with the images to label.\n Supported image formats are: `jpg`, `jpeg`, `png`.\n The `config.yaml` file creates a *Points layer*, which holds metadata (such as keypoints read from the config file) necessary for labeling.\n Select the *Points layer* in the layer list (lower left pane on the GUI) and click on the *+*-symbol in the layer controls menu (upper left pane) to start labeling.\n The current keypoint can be viewed/selected in the keypoints dropdown menu (right pane).\n The slider below the displayed image (or the left/right arrow keys) allows selecting the image to label.\n\n To save the labeling progress refer to [Save Layers](#save-layers).\n `Data successfully saved` should be shown in the status bar, and the image folder should now contain a `CollectedData_.h5` file.\n (Note: For convenience, a CSV file with the same name is also saved.)\n\n2. **Resuming labeling** – the image folder contains a `CollectedData_.h5` file.\n\n Open *napari* and open an image folder (which needs to contain a `CollectedData_.h5` file).\n In this case, it is not necessary to open the DLC project's `config.yaml` file, as all necessary metadata is read from the `h5` data file.\n\n Saving works as described in *1*.\n\n ***Note that if a new body part has been added to the `config.yaml` file after having started to label, loading the config in the GUI is necessary to update the dropdown menus and other metadata.***\n\n ***As `viridis` is `napari-deeplabcut` default colormap, loading the config in the GUI is also needed to update the color scheme.***\n\n3. **Refining labels** – the image folder contains a `machinelabels-iter<#>.h5` file.\n\n The process is analog to *2*.\n Open *napari* and open an image folder.\n If the video was originally labeled, *and* had outliers extracted it will contain a `CollectedData_.h5` file and a `machinelabels-iter<#>.h5` file. In this case, select the `machinelabels` layer in the GUI, and type `e` to show edges. Red indicates likelihood < 0.6. As you navigate through frames, images with labels with edges will need to be refined (moved, deleted, etc). Images with labels without edges will be on the `CollectedData` (previous manual annotations) layer and shouldn't need refining. However, you can switch to that layer and fix errors. You can also right-click on the `CollectedData` layer and select `toggle visibility` to hide that layer. Select the `machinelabels` layer before saving which will append your refined annotations to `CollectedData`.\n\n If the folder only had outliers extracted and wasn't originally labeled, it will not have a `CollectedData` layer. Work with the `machinelabels` layer selected to refine annotation positions, then save.\n\n In this case, it is not necessary to open the DLC project's `config.yaml` file, as all necessary metadata is read from the `h5` data file.\n\n Saving works as described in *1*.\n\n4. **Drawing segmentation masks**\n\n Drop an image folder as in *1*, manually add a *shapes layer*. Then select the *rectangle* in the layer controls (top left pane),\n and start drawing rectangles over the images. Masks and rectangle vertices are saved as described in [Save Layers](#save-layers).\n Note that masks can be reloaded and edited at a later stage by dropping the `vertices.csv` file onto the canvas.\n\n### Workflow flowchart\n\n```mermaid\n%%{init: {\"flowchart\": {\"htmlLabels\": false}} }%%\ngraph TD\n id1[What stage of labeling?]\n id2[deeplabcut.label_frames]\n id3[deeplabcut.refine_labels]\n id4[Add labels to, or modify in, \\n `CollectedData...` layer and save that layer]\n id5[Modify labels in `machinelabels` layer and save \\n which will create a `CollectedData...` file]\n id6[Have you refined some labels from the most recent iteration and saved already?]\n id7[\"All extracted frames are already saved in `CollectedData...`.\n1. Hide or trash all `machinelabels` layers.\n2. Then modify in and save `CollectedData`\"]\n id8[\"\n1. hide or trash all `machinelabels` layers except for the most recent.\n2. Select most recent `machinelabels` and hit `e` to show edges.\n3. Modify only in `machinelabels` and skip frames with labels without edges shown.\n4. Save `machinelabels` layer, which will add data to `CollectedData`.\n\t- If you need to revisit this video later, ignore `machinelabels` and work only in `CollectedData`\"]\n\n id1 -->|I need to manually label new frames \\n or fix my labels|id2\n id1 ---->|I need to refine outlier frames \\nfrom analyzed videos|id3\n id2 -->id4\n id3 -->|I only have a `machinelabels...` file|id5\n id3 ---->|I have both `machinelabels` and `CollectedData` files|id6\n id6 -->|yes|id7\n id6 ---->|no, I just extracted outliers|id8\n```\n\n### Labeling multiple image folders\n\nLabeling multiple image folders has to be done in sequence; i.e., only one image folder can be opened at a time.\nAfter labeling the images of a particular folder is done and the associated *Points layer* has been saved, *all* layers should be removed from the layers list (lower left pane on the GUI) by selecting them and clicking on the trashcan icon.\nNow, another image folder can be labeled, following the process described in *1*, *2*, or *3*, depending on the particular image folder.\n\n\n### Defining cropping coordinates\n\nPrior to defining cropping coordinates, two elements should be loaded in the GUI:\na video and the DLC project's `config.yaml` file (into which the crop dimensions will be stored).\nThen it suffices to add a `Shapes layer`, draw a `rectangle` in it with the desired area,\nand hit the button `Store crop coordinates`; coordinates are automatically written to the configuration file.\n\n\n## Contributing\n\nContributions are very welcome. Tests can be run with [tox], please ensure\nthe coverage at least stays the same before you submit a pull request.\n\nTo locally install the code, please git clone the repo and then run `pip install -e .`\n\n## License\n\nDistributed under the terms of the [BSD-3] license,\n\"napari-deeplabcut\" is free and open source software.\n\n## Issues\n\nIf you encounter any problems, please [file an issue] along with a detailed description.\n\n[file an issue]: https://github.com/DeepLabCut/napari-deeplabcut/issues\n\n\n## Acknowledgements\n\n\nThis [napari] plugin was generated with [Cookiecutter] using [@napari]'s [cookiecutter-napari-plugin] template. We thank the Chan Zuckerberg Initiative (CZI) for funding the initial development of this work!\n\n\n\n\n[napari]: https://github.com/napari/napari\n[Cookiecutter]: https://github.com/audreyr/cookiecutter\n[@napari]: https://github.com/napari\n[cookiecutter-napari-plugin]: https://github.com/napari/cookiecutter-napari-plugin\n[BSD-3]: http://opensource.org/licenses/BSD-3-Clause\n[tox]: https://tox.readthedocs.io/en/latest/\n[pip]: https://pypi.org/project/pip/\n[PyPI]: https://pypi.org/\n","description_content_type":"text/markdown","keywords":null,"home_page":"https://github.com/DeepLabCut/napari-deeplabcut","download_url":null,"author":"Team DeepLabCut, Lead by Jessy Lauer","author_email":"admin@deeplabcut.org","maintainer":null,"maintainer_email":null,"license":"BSD-3-Clause","classifier":["Development Status :: 2 - Pre-Alpha","Framework :: napari","Intended Audience :: Developers","License :: OSI Approved :: BSD License","Operating System :: OS Independent","Programming Language :: Python :: 3","Programming Language :: Python :: 3 :: Only","Programming Language :: Python :: 3.8","Programming Language :: Python :: 3.9","Programming Language :: Python :: 3.10","Topic :: Scientific/Engineering :: Artificial Intelligence","Topic :: Scientific/Engineering :: Image Processing","Topic :: Scientific/Engineering :: Visualization"],"requires_dist":["dask-image","matplotlib >=3.3","napari ==0.4.18","natsort","numpy","opencv-python-headless","pandas","pyyaml","qtpy >=2.4","scikit-image","tables","napari ; extra == 'testing'","pyside6 <6.3.2 ; extra == 'testing'","pytest ; extra == 'testing'","pytest-cov ; extra == 'testing'","pytest-qt ; extra == 'testing'","tox ; extra == 'testing'"],"requires_python":">=3.8","requires_external":null,"project_url":["Bug Tracker, https://github.com/DeepLabCut/napari-deeplabcut/issues","Documentation, https://github.com/DeepLabCut/napari-deeplabcut#README.md","Source Code, https://github.com/DeepLabCut/napari-deeplabcut","User Support, https://github.com/DeepLabCut/napari-deeplabcut/issues"],"provides_extra":["testing"],"provides_dist":null,"obsoletes_dist":null},"npe1_shim":false}