pipeline.src.flows.anchorages ============================= .. py:module:: pipeline.src.flows.anchorages Classes ------- .. autoapisummary:: pipeline.src.flows.anchorages.PortLocation pipeline.src.flows.anchorages.PortsVPTree Functions --------- .. autoapisummary:: pipeline.src.flows.anchorages.extract_ports pipeline.src.flows.anchorages.extract_control_ports_locodes pipeline.src.flows.anchorages.extract_ers_ports_locodes pipeline.src.flows.anchorages.extract_ais_anchorage_coordinates pipeline.src.flows.anchorages.extract_vms_static_positions pipeline.src.flows.anchorages.extract_manual_anchorages_coordinates pipeline.src.flows.anchorages.get_anchorage_h3_cells pipeline.src.flows.anchorages.get_anchorage_h3_cells_rings pipeline.src.flows.anchorages.get_ports_locations pipeline.src.flows.anchorages.get_anchorages_closest_port pipeline.src.flows.anchorages.unite_ports_locodes pipeline.src.flows.anchorages.get_active_ports pipeline.src.flows.anchorages.merge_closest_port_closest_active_port pipeline.src.flows.anchorages.load_processed_anchorages pipeline.src.flows.anchorages.anchorages_compute_flow pipeline.src.flows.anchorages.extract_datagouv_anchorages pipeline.src.flows.anchorages.load_anchorages_to_monitorfish pipeline.src.flows.anchorages.anchorages_flow Module Contents --------------- .. py:class:: PortLocation .. py:attribute:: locode :type: str .. py:attribute:: port_name :type: str .. py:attribute:: latitude :type: float .. py:attribute:: longitude :type: float .. py:class:: PortsVPTree(ports_locations: List[PortLocation]) Bases: :py:obj:`vptree.VPTree` Vantage Point Tree to efficiently find the nearest port from a given Position(lat, lon). If there are p ports in the tree, searching for the port that is closest to a given Position has complexity log(p). .. py:method:: get_nearest_port(pos: src.helpers.spatial.Position) -> dict Returns the distance (in meters) and locode of the PortLocation that is closest to the input Position. :param pos: Position instance :type pos: Position :returns: dict with nearest_port_distance and nearest_port_locode keys. :rtype: dict .. py:function:: extract_ports() -> pandas.DataFrame Extracts ports locode, name, latitude and longitude from processed.ports. This table therefore needs to be filled before using this function. :returns: DataFrame of ports with locode, port_name, longitude and latitude columns. :rtype: pd.DataFrame .. py:function:: extract_control_ports_locodes() Returns the set of distinct port locodes where at least one control was done. :returns: set of port locodes :rtype: Set[str] .. py:function:: extract_ers_ports_locodes() -> Set[str] Returns the set of distinct port locodes used at least once in an ERS DEP, PNO ou LAN message. :returns: set of port locodes :rtype: Set[str] .. py:function:: extract_ais_anchorage_coordinates() -> pandas.DataFrame Returns a DataFrame with latitude, longitude columns corresponding to S2 cells identified as docks in AIS global positions. .. py:function:: extract_vms_static_positions(parquet_file_relative_path) -> pandas.DataFrame Read local file with vms positions that have speed zero. :returns: DataFrame with latitude and longitude columns. :rtype: pd.DataFrame .. py:function:: extract_manual_anchorages_coordinates() -> pandas.DataFrame .. py:function:: get_anchorage_h3_cells(static_positions: pandas.DataFrame, h3_resolution: int = 9, number_signals_threshold: int = 100) -> pandas.DataFrame Bins input positions into h3 cells of the given resolutions and filters said h3 cells to keep only the ones that appear at least `number_signals_threshold` times in the dataset. :param static_positions: DataFrame with latitude and longitude columns :type static_positions: pd.DataFrame :param h3_resolution: h3 resolution to use :type h3_resolution: int :param number_signals_threshold: number of occurences below which h3 cells are filtered out :type number_signals_threshold: int .. py:function:: get_anchorage_h3_cells_rings(ais_anchorage_h3_cells: Set[str], vms_anchorage_h3_cells: Set[str], manual_anchorage_h3_cells: Set[str]) -> pandas.DataFrame Unites two sets of h3 cells corresponding to anchorage locations of vessels in AIS and VMS data, then adds two "rings" of cells around them. Returns the result as a DataFrame containing the indices, latitude and longitude of cells as well as whether each cell was present in the original cells (ring 0) or was added in rings 1 and 2 that surround the initial cells. :param ais_anchorage_h3_cells: set of indices of h3 cells where vessels anchor (AIS data) :type ais_anchorage_h3_cells: Set[str] :param vms_anchorage_h3_cells: set of indices of h3 cells where vessels anchor (VMS data) :type vms_anchorage_h3_cells: Set[str] :param manual_anchorage_h3_cells: set of additional indices of h3 cells :type manual_anchorage_h3_cells: Set[str] :returns: DataFrame of h3 cells with 2 levels of rings added :rtype: pd.DataFrame .. py:function:: get_ports_locations(ports: pandas.DataFrame) -> List[PortLocation] Transforms a DataFrame into a list of PortLocation objects. :param ports: DataFrame with columns matching the fields of a PortLocation object. :type ports: pd.DataFrame :returns: List[PortLocation] .. py:function:: get_anchorages_closest_port(anchorage_h3_cells_rings: pandas.DataFrame, ports_locations: List[PortLocation]) -> pandas.DataFrame .. py:function:: unite_ports_locodes(ers_ports_locode: Set[str], control_ports_locodes: Set[str]) -> Set[str] Unites sets of port locodes. :param ers_ports_locode: set of the locodes of ports used in ERS :type ers_ports_locode: Set[str] :param control_ports_locodes: set of the locodes of ports used in controls :type control_ports_locodes: Set[str] :returns: union of the two input sets :rtype: Set[str] .. py:function:: get_active_ports(ports: pandas.DataFrame, active_ports_locodes: Set[str]) -> pandas.DataFrame .. py:function:: merge_closest_port_closest_active_port(anchorages_closest_port: pandas.DataFrame, anchorages_closest_active_port: pandas.DataFrame) -> pandas.DataFrame Merges anchorages closest port and closest active port. .. py:function:: load_processed_anchorages(anchorages: pandas.DataFrame) Load anchorages to processed.anchorages .. py:function:: anchorages_compute_flow(h3_resolution: int = ANCHORAGES_H3_CELL_RESOLUTION, number_signals_threshold: int = 100, static_vms_positions_file_path: str = 'data/raw/anchorages/static_vms_positions_2021_03_to_10.parquet') Flow to compute anchorages and attribute cells to ports .. py:function:: extract_datagouv_anchorages(anchorages_url: str, proxies: dict) -> pandas.DataFrame Downloads anchorages csv file, returns the result as a pandas DataFrame. :param anchorages_url: url to download the data from. :type anchorages_url: str :param proxies: dict with http_proxy and https_proxy settings to use for the download :type proxies: dict :returns: anchorages data :rtype: pd.DataFrame .. py:function:: load_anchorages_to_monitorfish(anchorages: pandas.DataFrame) Loads anchorages data to monitorfish database. :param anchorages: anchorages data :type anchorages: pd.DataFrame .. py:function:: anchorages_flow() Main anchorages flow - extract from data.gouv.fr and load to database