Speckle2Self: ultrasound speckle reduction

 

This notebook demonstrates how to perform self-supervised ultrasound speckle reduction using Speckle2Self within the zea framework. We apply the model to an in-vivo carotid artery scan from zeahub/zea-carotid-2023, matching the domain on which the speckle2self-invivo weights were trained.

For more information on the method, see the original paper:

• Lin, Huiping, et al. “Speckle2Self: Learning Self-Supervised Despeckling with Attention Mechanism for SAR Images.” Remote Sensing 17.23 (2025): 3840.

 

‼️ Important: This notebook is optimized for GPU/TPU. Code execution on a CPU may be very slow.

If you are running in Colab, please enable a hardware accelerator via:

Runtime → Change runtime type → Hardware accelerator → GPU/TPU 🚀.

%%capture
%pip install zea

import os

os.environ["KERAS_BACKEND"] = "tensorflow"

import numpy as np
import keras
import matplotlib.pyplot as plt

import zea
from zea.ops import (
    Beamform,
    EnvelopeDetect,
    Pipeline,
    Cast,
    Demodulate,
    Downsample,
    Normalize,
    LogCompress,
)
from zea.visualize import set_mpl_style

zea: Using backend 'tensorflow'

zea.init_device(verbose=False)
set_mpl_style()

carotid_path = "hf://zeahub/picmus/in_vivo/carotid_cross/carotid_cross_expe_dataset_rf/carotid_cross_expe_dataset_rf.hdf5"
frame_idx = 0
n_tx = 11  # transmits per frame (fewer -> faster; more -> better quality)
dynamic_range = (-40, 0)  # dB

Load carotid data

We load a single frame of an in-vivo carotid scan from zeahub/zea-carotid-2023 on Hugging Face. Note that this is raw ultrasound RF channel data. Therefore we will first need to beamform and envelope-detect the data before feeding it to the Speckle2Self model. We will do this using the zea.Pipeline framework, which provides efficient implementations of these operations. For more details see the pipeline example notebook or the API reference.

with zea.File(carotid_path, revision="v0.1.0") as f:
    data = f.data.raw_data[frame_idx]
    parameters = f.load_parameters()

parameters.set_transmits(n_tx)
parameters.zlims = (0, 0.04)
parameters.grid_size_x = 500
parameters.grid_size_z = 800

pipeline = Pipeline(
    operations=[
        Cast(dtype="float32"),
        Demodulate(),
        Downsample(factor=2),
        Beamform(
            beamformer="delay_and_sum",
            enable_pfield=True,
            num_patches=100,
        ),
        EnvelopeDetect(),
        Normalize(),
        LogCompress(),
    ],
    with_batch_dim=False,
    jit_options="pipeline",
)

inputs = pipeline.prepare_parameters(parameters, dynamic_range=dynamic_range)

zea: Running compute_pfield and caching the result to /root/.cache/zea/cached_funcs/compute_pfield_3b656eda410eab8a0941a76c49516a15.pkl.

# Beamform + envelope-detect
data = data[parameters.selected_transmits]
out = pipeline(data=data, **inputs)
envelope = out[pipeline.output_key]
envelope = keras.ops.convert_to_numpy(envelope)

Load Speckle2Self model

Speckle2Self can be loaded directly from Hugging Face:

from zea.models.speckle2self import Speckle2Self

model = Speckle2Self.from_preset("hf://zeahub/speckle2self-invivo")

Preprocess and run inference

# Run despeckling (Speckle2Self) model inference
despeckled = model(envelope[None, ..., None])
despeckled = keras.ops.convert_to_numpy(despeckled)
despeckled = despeckled.squeeze()

Visualize results

For a simple comparison, we display one frame before and after Speckle2Self despeckling.

xlims_mm = [v * 1e3 for v in parameters.xlims]
zlims_mm = [v * 1e3 for v in parameters.zlims]
extent = [xlims_mm[0], xlims_mm[1], zlims_mm[1], zlims_mm[0]]

# gamma correction for better visualization
despeckled_viz = np.power(despeckled, 1.3)

fig, axes = plt.subplots(1, 2, figsize=(8, 4), squeeze=False)

axes[0, 0].imshow(envelope, cmap="gray", extent=extent)
axes[0, 0].set_title("B-Mode")
axes[0, 0].set_xlabel("X (mm)")
axes[0, 0].set_ylabel("Z (mm)")

axes[0, 1].imshow(despeckled_viz, cmap="gray", extent=extent)
axes[0, 1].set_title("Despeckled B-Mode (Speckle2Self)")
axes[0, 1].set_xlabel("X (mm)")
axes[0, 1].set_ylabel("Z (mm)")

plt.tight_layout()
plt.savefig("speckle2self_output.png", bbox_inches="tight", dpi=100)
plt.close()

Speckle2Self result on in-vivo carotid data