Speed up HSM plugins in Python

Description

HSM plugins as implemented in Python does some deep copying of data more than it needs to, slowing down the process tremendously. This is (one of) the reason(s) for the ci_imsim slowdown reported in DM-41865,

Attachments

Linked work items

is triggered by

DM-30947

Reimplement C++ HSM moment measurement algorithms in Python

relates to

DM-41865

ci_imsim got slower on jenkins runs from 2023-11-23

DM-41866

verify_drp_metrics is taking 50% longer since d_2023_11_24

Confluence content

mentioned on

https://rubinobs.atlassian.net/wiki/pages/viewpage.action?pageId=48835439

Issue Matrix

hide

Activity

Show:

Arun Kannawadi

November 29, 2023 at 5:48 PM

Jenkins is running: http://rubin-ci.slac.stanford.edu/job/stack-os-matrix/27/display/redirect

Eli Rykoff

November 29, 2023 at 5:46 PM

Also related to https://rubinobs.atlassian.net/browse/DM-41866#icft=DM-41866. Will close https://rubinobs.atlassian.net/browse/DM-41865#icft=DM-41865 and https://rubinobs.atlassian.net/browse/DM-41866#icft=DM-41866 when this is merged and we have a nightly run to confirm.

Eli Rykoff

November 29, 2023 at 5:41 PM

Great teamwork on this!

Arun Kannawadi

November 29, 2023 at 5:37 PM

Thanks @Eli Rykoff, @Nate Lust, @Jim Bosch and @Erfan Nourbakhsh for all the help in party-programming and finding the issues.

Resize issue view side panel

Done

Pinned fields

Click on the next to a field label to start pinning.

Details

Assignee

Arun Kannawadi

Reporter

Arun Kannawadi

Labels

Reviewers

Eli Rykoff

Story Points

RubinTeam

External

Components

meas_extensions_shapeHSM

Checklist

Created November 29, 2023 at 5:04 PM

Updated July 15, 2024 at 7:42 PM

Resolved November 30, 2023 at 3:11 AM