Model Construction Extension API

Extension point. Builders define the model names and families reachable from canonical config. Concrete modules are internal implementations.

Dispatch rules

models.builder.build_model() accepts canonical runtime config and dispatches by TASK. Classification selects an encoder route. Localization selects MODEL.metadata.family (RuleBased, E2E, ContextAware, or LearnablePooling). VQA selects its configured backend (xvars_videochatgpt, qwen_xvars_infer, or qwen_vl_native_infer). An unsupported name raises ValueError; it is not dynamically imported.

Builder	Responsibility
models.builder	Top-level canonical model route.
models.backbones.builder	Video, feature, graph, and image backbone construction.
models.neck.builder	Aggregation and temporal/pooling adapters.
models.heads.builder	Classification and spotting output heads.

Model builder entrypoints.

build_model_canonical(config, device)

Build model directly from canonical config.

Source code in opensportslib/models/builder.py

def build_model_canonical(config, device):
    """Build model directly from canonical config."""
    task = config.TASK.lower()

    encoder_type = get_component_name_by_kind(config, "encoder")
    sampling = get_data_sampling(config)
    model_family = get_model_family(config)

    if task == "classification":
        backbone = _component_cfg(config, "encoder")
        head = _component_cfg(config, "head")
        if encoder_type == "video_mae":
            from opensportslib.models.base.video import build_video_mae_backbone
            return build_video_mae_backbone(config, device)

        elif encoder_type in ["r3d_18", "mc3_18", "r2plus1d_18", "s3d", "mvit_v2_s"]:
            from opensportslib.models.base.vars import MVNetwork
            neck = _component_cfg(config, "adapter")
            return MVNetwork(config, backbone, neck, head), None

        elif encoder_type == "graph_conv":
            from opensportslib.models.base.tracking import TrackingModel
            return TrackingModel(config, device), None

        elif encoder_type in ("dinov3", "clip", "videomae", "videomae2"):
            from opensportslib.models.base.video import VideoModel
            return VideoModel(config, device), None

        else:
            raise ValueError(f"Unsupported encoder type: {encoder_type}")

    if task == "localization":
        from opensportslib.models.base.e2e import E2EModel
        from opensportslib.models.base.contextaware import LiteContextAwareModel
        from opensportslib.models.base.learnablepooling import LiteLearnablePoolingModel

        model_weights = (
            get_model_load(config).get("checkpoint_path")
            or get_component_load_by_kind(config, "encoder").get("weights_path")
        )
        runner = get_runner_type(config)
        normalized_family = str(model_family or "").strip().lower()

        if normalized_family == "rulebased":
            from opensportslib.models.base.rule_based import build_rule_based_model
            return build_rule_based_model(config)

        backbone = _component_cfg(config, "encoder")
        head = _component_cfg(config, "head")

        if normalized_family == "learnablepooling":
            neck = _component_cfg(config, "adapter")
            post_proc = _component_cfg(config, "postprocessor", required=False)
            return LiteLearnablePoolingModel(
                cfg=config,
                weights=model_weights,
                backbone=backbone,
                head=head,
                neck=neck,
                post_proc=post_proc,
                runner=runner,
            )
        if normalized_family == "contextaware":
            neck = _component_cfg(config, "adapter")
            return LiteContextAwareModel(
                cfg=config,
                weights=model_weights,
                backbone=backbone,
                head=head,
                neck=neck,
                runner=runner,
            )

        if normalized_family == "e2e":
            return E2EModel(
                config,
                get_data_num_classes(config) + 1,
                backbone,
                head,
                clip_len=sampling.get("clip_len"),
                modality=get_runtime_modality(config),
                device=device,
                multi_gpu=getattr(config.TRAIN.execution, "multi_gpu", False),
            )

        raise ValueError(
            f"Unsupported localization model family: {model_family!r}. "
            "Expected one of: E2E, ContextAware, LearnablePooling."
        )
    if task == "vqa":
        backend = get_vqa_backend(config)

        if backend == "xvars_videochatgpt":
            from opensportslib.models.base.xvars_videochatgpt import XVarsVideoChatGPTModel

            projector_params = get_component_params_by_kind(config, "projector")
            model_id = get_vqa_decoder_model_id(config, default="base_model_videoChatGPT")
            return XVarsVideoChatGPTModel(
                config,
                model_id=model_id,
                projector_params=projector_params,
            ), None
        if backend == "qwen_xvars_infer":
            from opensportslib.models.base.qwen_xvars import QwenXVarsModel

            projector_params = get_component_params_by_kind(config, "projector")
            model_id = get_vqa_decoder_model_id(config, default="Qwen/Qwen3.5-9B-Base")
            return QwenXVarsModel(
                config,
                model_id=model_id,
                projector_params=projector_params,
            ), None
        if backend == "qwen_vl_native_infer":
            from opensportslib.models.base.qwen_vl_native import QwenVLNativeModel

            model_id = get_vqa_decoder_model_id(config, default="Qwen/Qwen2.5-VL-7B-Instruct")
            return QwenVLNativeModel(
                config,
                model_id=model_id,
            ), None

        raise ValueError(
            f"Unsupported VQA backend '{backend}'. "
            "Only 'xvars_videochatgpt', 'qwen_xvars_infer', and 'qwen_vl_native_infer' are supported."
        )
    else:
        raise ValueError(f"Unsupported model family for task: {task}")

build_model_from_config(config, device)

Version-neutral public dispatcher for model construction.

Source code in opensportslib/models/builder.py

def build_model_from_config(config, device):
    """Version-neutral public dispatcher for model construction."""
    model_cfg = getattr(config, "MODEL", None)
    if hasattr(model_cfg, "components"):
        return build_model_canonical(config, device)
    raise ValueError("Only canonical model config is supported at runtime.")

build_model(config, device)

Backward-compatible alias for the public dispatcher.

Source code in opensportslib/models/builder.py

def build_model(config, device):
    """Backward-compatible alias for the public dispatcher."""
    return build_model_from_config(config, device)

Namespace-module builders

models/backbones, models/neck, and models/heads are namespace-style source directories in this repository, so MkDocstrings cannot resolve them as Python packages. Their concrete extension signatures are:

build_backbone(cfg, default_args=None)
Add_Temporal_Shift_Modules(net, n_segment, is_gsm=False)
build_neck(cfg, default_args=None)
build_head(cfg, default_args=None)

The source modules are models/backbones/builder.py, models/neck/builder.py, and models/heads/builder.py. They remain extension points, not public imports.

Concrete task models

Internal. These classes are the concrete implementations selected by the top-level builder. They are cataloged for maintainers, not offered as stable application imports.

Task / family	Module	Classes and functions	Builder selection
Classification multi-view	models/base/vars.py	MVNetwork	Video encoder route such as Torchvision video models.
Classification tracking	models/base/tracking.py	TrackingModel	encoder name graph_conv.
Classification video	models/base/video.py	VideoModel, build_video_mae_backbone, load_video_mae_checkpoint	Feature-extractor/video route; video_mae uses the dedicated builder.
Localization E2E	models/base/e2e.py	E2EModel	MODEL.metadata.family: E2E.
Localization context-aware	models/base/contextaware.py	ContextAwareModel, LiteContextAwareModel	ContextAware.
Localization pooling	models/base/learnablepooling.py	LearnablePoolingModel, LiteLearnablePoolingModel	LearnablePooling.
Rule-based HDF5 spotting	models/base/rule_based.py	H5 JSON dataset plus header/skeleton spotter variants; build_rule_based_model	RuleBased.
X-VARS VQA	models/base/xvars_videochatgpt.py	XVarsVideoChatGPTCausalLM, raw-video feature extractors, XVarsVideoChatGPTModel	xvars_videochatgpt.
Qwen-XVARS VQA	models/base/qwen_xvars.py	QwenXVarsCausalLM, QwenXVarsModel	qwen_xvars_infer.
Native Qwen VL	models/base/qwen_vl_native.py	QwenVLNativeModel, native collator/trainer/error classes	qwen_vl_native_infer.

The rule-based module contains these named spotter implementations: H5HeaderSpotter, distance/speed/angle variants, H5HeaderSkeletonSpotter, recall variants, and their common base classes. They are selected by canonical rule-variant configuration, not by direct public construction.

Backbone, neck, and head class catalog

Component area	Internal classes
Backbones	BaseExtractFeatures, ConvNextTinyExtractFeatures, RegnetyExtractFeatures, ResnetExtractFeatures, PreExtactedFeatures, TorchvisionVideoExtractFeatures, GraphEncoder, GraphSequenceEncoder, VideoBackbone
Necks/adapters	MVAggregate, WeightedAggregate, ViewMaxAggregate, ViewAvgAggregate, TemporalAggregation, CNN_temporally_aware, max/average pool variants, NetRVLAD, NetVLAD, and their temporal/core variants
Heads	MVHead, TrackingClassifierHead, TemporalE2EHead, LinearLayerHead, SpottingCALFHead

Backbone, neck, and head constructor details are intentionally read from the builder configuration and component params; their exact source signatures may change with the internal implementations.

Model utilities

Module	Non-private classes/functions	Purpose
models/utils/common.py	ABCModel, BaseRGBModel, step, SingleStageGRU, SingleStageTCN	Common model and optimization-loop support.
models/utils/litebase.py	LiteBaseModel	Lightning base for lite localization families.
models/utils/modules.py	FCPrediction, GRUPrediction, TCNPrediction, ASFormerPrediction	Temporal prediction blocks.
models/utils/shift.py	GatedShift, make_temporal_shift	Temporal shift/GSM support.
models/utils/xvars_clip_index.py	feature/prediction index loaders and tensor validator	VQA feature/index lookup.
models/utils/vqa_prompting.py, vqa_prediction_priors.py	prompt/prior builders	VQA prompt construction.
models/utils/utils.py	NMS, prediction JSON, timestamp, and result helpers	Localization prediction formatting; some helpers write archives/JSON.
models/utils/impl/	CALF, ASFormer, GTAD, TSM/GSM implementation classes	Algorithm-specific internal layers.

Utilities that write prediction JSON, zip results, or load feature files have filesystem side effects. VQA feature/index utilities require the referenced local artifacts to exist; model constructors may additionally download Hub weights when their configured source requests them.

To add a component, first give it a canonical component name and parameters, then route it through the matching builder and prove it with a config and test. Do not expose a concrete nn.Module as a stable user API merely because a builder uses it.