fastEmbedR: native UMAP and t-SNE workflows
Stefano Cacciatore
Source:vignettes/getting-started.Rmd
getting-started.RmdOverview
fastEmbedR provides native implementations of two widely
used nonlinear dimension-reduction workflows:
- UMAP from a sparse nearest-neighbor graph;
- interpolation-based t-SNE with sparse attractive forces and an FFT-grid approximation for repulsion.
The same R interface selects native CPU, Apple Metal, or NVIDIA CUDA code. One-call functions include nearest-neighbor search, initialization, graph or affinity construction, and optimization. KNN-input functions let an existing neighbor graph be reused without repeating search. Explicit GPU requests either run on the requested GPU backend or fail with an informative error; they are never silently reported as GPU work after falling back to CPU.
Select the backend and CPU core count
Every backend-capable function accepts backend = "cpu",
"metal", or "cuda". An explicit function
argument is the clearest choice for a reproducible script:
fit_cpu <- fastEmbedR::umap(
x,
backend = "cpu",
n.cores = 4,
seed = 4
)
tsne_cpu <- fastEmbedR::tsne(
x,
perplexity = 10,
backend = "cpu",
n.cores = 4,
seed = 4
)
fit_metal <- fastEmbedR::umap(x, backend = "metal", seed = 4)
fit_cuda <- fastEmbedR::umap(x, backend = "cuda", seed = 4)n.cores is a positive integer controlling CPU parallel
work, including the native nearest-neighbor search and the CPU graph,
affinity, and optimization stages that support parallel execution. For
UMAP, the optimizer currently uses at most four workers and records the
requested and effective values. When n.cores is omitted,
options(n.cores = ...) is used when set. Otherwise KNN and
PCA use one worker, UMAP selects a size-aware default from one to four
workers, and t-SNE uses four workers. Metal and CUDA execute on their
respective accelerators and ignore n.cores; the argument
does not control GPU concurrency.
For an interactive session, set defaults once and omit
backend and n.cores from later calls:
old_options <- options(backend = "cpu", n.cores = 4L)
u <- fastEmbedR::umap(x)
t <- fastEmbedR::tsne(x, perplexity = 10)
u.cpu <- fastEmbedR::umap( # explicit arguments take precedence
x,
backend = "cpu",
n.cores = 2
)
options(old_options)Set options(backend = "metal") or
options(backend = "cuda") for an accelerator session. For a
batch job, set FASTEMBEDR_BACKEND before starting R, for
example FASTEMBEDR_BACKEND=cuda Rscript analysis.R.
Resolution precedence is:
- the explicit
backendfunction argument; -
options(backend = ...); - the
FASTEMBEDR_BACKENDenvironment variable; - CPU when none of the above is set.
The backend requested by the user and the backend that actually ran
are also recorded in returned objects, for example
fit_cpu$parameters[c("backend", "nn_backend", "n.cores")].
A requested Metal or CUDA backend must be available and execute
natively; otherwise the function stops with an informative error instead
of silently using CPU.
The separate gpuinfo package reports host GPU, driver,
and toolkit visibility. For example, gpuinfo::gpu_sitrep()
helps diagnose an unavailable accelerator. Hardware visibility alone
does not establish that fastEmbedR was compiled for it; confirm
execution with the returned backend metadata from an explicit
backend = "metal" or backend = "cuda"
call.
The public workflow is intentionally small:
| Goal | Function |
|---|---|
| Precompute native non-self neighbors | precompute_knn() |
| Run UMAP from a data matrix | umap() |
| Run interpolation-based t-SNE from a data matrix | tsne() |
| Reuse a KNN graph |
umap_knn() or tsne_knn()
|
| Compute reusable PCA scores or t-SNE initialization | pca() |
| Approximate a very large run |
landmarks in tsne() or
umap()
|
| Place new observations in a fixed t-SNE reference | transform_tsne() |
| Quantify embedding quality | evaluate_embedding() |
Because other R packages export generic names such as
umap() and pca(), use
fastEmbedR::umap() and fastEmbedR::pca() in
scripts, package code, and published analyses. fastEmbedR follows
ordinary R search-path rules and does not silently resolve masking
conflicts.
A first analysis
We use the four numeric Iris measurements for a small, reproducible
example. Iris is supplied by R’s separate datasets package
and is not bundled with fastEmbedR. The matrix is scaled once outside
the embedding calls, so standardize = FALSE prevents
repeated preprocessing.
library(fastEmbedR)
iris_data <- datasets::iris
x <- scale(as.matrix(iris_data[, 1:4]))
labels <- iris_data$Species
dim(x)
#> [1] 150 4UMAP uses a neighborhood size, whereas t-SNE uses perplexity. For this small dataset we use 15 neighbors and perplexity 10. Missing t-SNE optimization settings are chosen by the package’s native auto-configuration policy. Its default initialization is a small-scale randomized-PCA layout.
fit_umap <- umap(
x,
n_neighbors = 15,
graph_mode = "fuzzy",
standardize = FALSE,
n.cores = 2,
seed = 4
)
fit_tsne <- tsne(
x,
perplexity = 10,
standardize = FALSE,
n.cores = 2,
seed = 4
)
c(
UMAP_seconds = fit_umap$metrics$elapsed,
tsne_seconds = fit_tsne$metrics$elapsed
)
#> UMAP_seconds tsne_seconds
#> 0.165 14.424Both functions return a fastEmbedR_embedding object. The
full coordinates are in $layout; $metrics,
$timings, and $parameters preserve enough
context to audit a run.
old_par <- par(mfrow = c(1, 2), mar = c(1, 1, 2.5, 1), bg = "white")
plot_embedding(fit_tsne$layout, labels, "t-SNE")
plot_embedding(fit_umap$layout, labels, "UMAP (fuzzy graph)")
Iris one-call CPU embeddings.
par(old_par)The backend recorded in the result is the backend that actually ran:
fit_umap$parameters[c("backend", "nn_backend", "n.cores", "graph_mode")]
#> $backend
#> [1] "cpu"
#>
#> $nn_backend
#> [1] "native_cpu_exact"
#>
#> $n.cores
#> [1] 2
#>
#> $graph_mode
#> [1] "fuzzy"
fit_tsne$parameters[c(
"backend", "nn_backend", "n.cores", "perplexity",
"affinity_support", "affinity_support_k"
)]
#> $backend
#> [1] "cpu"
#>
#> $nn_backend
#> [1] "cpu"
#>
#> $n.cores
#> [1] 2
#>
#> $perplexity
#> [1] 10
#>
#> $affinity_support
#> [1] "compact"
#>
#> $affinity_support_k
#> [1] 10Parameter philosophy and intended scope
The two embedding functions deliberately expose different levels of
control. t-SNE exposes perplexity/support, initialization, iteration
counts, early and normal exaggeration, learning rate, and momentum.
Repulsion uses FFT for both 2D and 3D output. Set
auto_config = FALSE when automatic iteration or stopping
choices are not appropriate.
UMAP is an opinionated high-throughput implementation, not a
general-purpose hyperparameter interface. Users select
n_neighbors, metric, graph mode, preprocessing, backend,
seed, and CPU thread count. The release policy selects the remaining
values and records them in fit$parameters: 500 epochs below
10,000 observations and 200 for larger ordinary profiles (at least 300
for high-variability profiles), five negative samples, spread 1,
repulsion strength 1, min_dist 0.01 or 0.1 under the
wide-shell rule, and learning rate 1 or 1.25 under the same rule.
umap_init() makes package-native initialization
reusable, but umap() does not accept arbitrary initial
coordinates. The nearest-neighbor router targets recall 0.99 and
intentionally hides index tuning; supply an external KNN object when
search policy must be controlled independently. Fixed seeds do not
ensure bitwise Metal or CUDA identity because atomic update order can
differ.
These settings affect cluster compactness, convergence effort, and
the balance between attractive and repulsive updates. Fixing them keeps
one tested policy aligned across CPU, Metal, and CUDA; it does not imply
that the selected values are universally optimal. Analyses requiring
min_dist, spread, epoch, learning-rate, repulsion,
negative-sample, or optimizer-mode sweeps should use a general-purpose
implementation such as uwot or umap-learn.
Float32 input, reusable KNN, prepared graphs, compact t-SNE support, and
landmarking remain explicit memory/speed controls; compact support and
landmarking must be described as approximations.
Choosing between UMAP and t-SNE
The two methods answer related but different questions.
| Property | UMAP | interpolation-based t-SNE |
|---|---|---|
| Model | Weighted or binary KNN graph | Sparse affinities |
| Main scale parameter | n_neighbors |
perplexity |
| Initialization | Graph-native initialization | Randomized PCA by default |
| Repulsion | Sampled negative edges | Exact or FFT-grid negative gradient |
| New data | Landmark projection | Fixed-reference transform |
UMAP emphasizes local neighborhoods with some graph-level organization. t-SNE emphasizes local probability neighborhoods and cluster organization.
Distances and empty space in either two-dimensional plot should not be treated as calibrated biological effect sizes. Compare methods with neighborhood and label-aware metrics rather than visual appearance alone.
Optional downstream graph clustering
Graph clustering is a secondary downstream utility; the primary package contribution is native UMAP and t-SNE embedding.
knn_graph() builds a compact undirected graph from raw
data, an embedding, or precomputed neighbors. It uses the existing KNN
route selected by backend; there is no second
nearest-neighbor method menu inside the graph API.
graph <- knn_graph(fit_tsne, k = 20, weight = "snn", n.cores = 2)
communities <- graph_cluster(graph, method = "leiden", seed = 4)
table(communities$membership)
#>
#> 1 2 3 4 5 6
#> 32 18 18 32 21 29Louvain and Leiden have native CPU, CUDA, and Metal backends and use
sparse graph storage. Select an accelerator explicitly with
graph_cluster(..., backend = "cuda") or
"metal"; an unavailable backend fails rather than falling
back to CPU. walktrap is the true Pons-Latapy random-walk
agglomeration method and remains CPU-only because its exact transition
matrix is intentionally limited to moderate graphs. Neither
igraph, cuGraph, nor an external clustering routine is
called at run time.
PCA and reproducible t-SNE initialization
pca() exposes the backend-native truncated PCA used for
initialization. It deliberately omits an SVD-method menu: randomized SVD
is the supported decomposition family. For backend = "cpu",
n.cores sets the CPU BLAS/OpenMP thread limit for the call.
tsne_init = FALSE is the default; request it when the
t-SNE-ready layout is needed. irlba is not used by the
package implementation and appears only as an external benchmark
reference.
pca_fit <- pca(
x,
ncomp = 2,
center = TRUE,
scale = FALSE,
backend = "cpu",
n.cores = 2,
seed = 4,
tsne_init = TRUE
)
c(
score_rows = nrow(pca_fit$scores),
score_columns = ncol(pca_fit$scores),
init_max_sd = max(apply(as.matrix(pca_fit$tsne_init), 2, stats::sd))
)
#> score_rows score_columns init_max_sd
#> 1.5e+02 2.0e+00 1.0e-04The ordinary scores remain in pca_fit$scores. The added
pca_fit$tsne_init is centered and rescaled so the largest
component standard deviation is 1e-4, the small
initialization scale expected by t-SNE optimizers. CPU fits record the
requested and observable thread counts in n.cores_requested
and n.cores_effective. A single-threaded BLAS may still
report one effective thread. The initialization can be reused across
methods, seeds, or backends:
layout_tsne_fixed_init <- tsne(
x,
perplexity = 10,
Y_init = pca_fit$tsne_init,
standardize = FALSE,
n.cores = 2,
seed = 4
)CPU PCA uses the package-native float32 blocked rSVD path. Metal uses
a native float32 block-subspace rSVD with Metal Performance Shaders
matrix products. CUDA always provides package-native rSVD and may select
RAPIDS RAFT TSVD for narrower or less-truncated problems when that
optional route is enabled. No backend calls Python or
reticulate. Float32 CUDA input is consumed without an
intermediate R double matrix, and float32 scores and loadings are
preserved.
Reusing a nearest-neighbor graph
The KNN-input API is useful when comparing embeddings, graph modes,
or seeds. precompute_knn() exposes the same
backend-specific native search selected by the one-call embedding
functions, without exposing a second algorithm menu.
knn <- precompute_knn(
x,
k = 30,
metric = "euclidean",
backend = "cpu",
n.cores = 2
)
knn
#> $indices
#> [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12] [,13] [,14] [,15]
#> [1,] 18 28 41 40 5 29 8 38 27 12 21 50 37 32 25
#> [2,] 26 13 46 35 10 31 4 36 3 30 48 39 9 50 14
#> [3,] 48 30 4 31 35 43 36 10 50 13 7 46 12 2 25
#> [4,] 48 31 30 3 39 43 35 13 46 10 2 14 9 26 36
#> [5,] 38 1 41 18 28 22 49 8 40 47 20 29 12 23 11
#> [6,] 17 45 19 20 47 11 49 22 15 33 34 5 37 28 18
#> [7,] 12 25 48 3 8 30 27 23 41 43 50 40 38 18 1
#> [8,] 40 50 29 12 1 27 41 18 25 28 5 24 36 38 21
#> [9,] 39 14 4 13 46 2 31 43 48 35 26 10 3 30 36
#> [10,] 35 31 13 2 26 36 30 46 3 4 48 50 39 43 8
#> [11,] 49 47 22 20 19 37 28 5 6 17 45 1 18 38 41
#> [12,] 25 8 7 50 27 40 41 1 18 30 29 3 38 48 5
#> [13,] 2 46 10 31 35 26 4 3 39 30 48 36 9 14 43
#> [14,] 39 9 4 43 48 13 46 3 31 30 2 10 35 26 36
#> [15,] 19 34 17 6 33 11 49 16 20 47 45 22 37 5 28
#> [16,] 34 15 33 6 17 19 20 45 47 11 49 22 5 37 38
#> [17,] 6 20 19 47 11 45 49 22 15 33 34 5 37 28 18
#> [18,] 1 41 28 40 5 29 8 27 38 44 32 12 21 22 50
#> [19,] 6 11 17 49 15 20 47 45 37 22 28 33 34 5 21
#> [20,] 47 45 22 49 11 6 17 5 38 18 1 41 28 19 33
#> [21,] 32 29 37 28 40 1 18 8 27 50 24 41 5 11 49
#> [22,] 20 45 47 49 5 11 18 41 38 1 28 44 6 17 27
#> [23,] 38 7 5 41 12 1 18 8 25 22 27 40 28 20 44
#> [24,] 27 50 32 40 8 44 29 36 18 21 12 41 25 28 1
#> [25,] 12 8 7 27 40 50 41 30 1 18 29 38 28 3 5
#> [26,] 2 35 10 13 46 31 36 4 30 3 48 50 39 9 24
#> [27,] 8 40 24 18 41 44 12 50 29 1 25 28 32 7 5
#> [28,] 1 18 29 40 41 8 5 21 37 32 27 38 49 11 50
#> [29,] 40 28 8 1 18 21 50 32 41 27 37 12 24 5 36
#> [30,] 48 3 31 4 35 10 43 36 50 12 25 13 7 46 2
#> [31,] 35 10 4 30 2 13 46 3 26 48 36 50 39 43 9
#> [32,] 21 29 37 28 40 18 24 27 1 8 41 44 50 5 36
#> [33,] 34 6 17 47 20 15 45 49 11 19 16 22 5 38 28
#> [34,] 33 16 15 17 6 19 47 20 45 11 49 22 5 38 37
#> [35,] 10 31 2 26 13 46 36 30 3 4 48 50 39 43 24
#> [36,] 50 35 10 3 31 30 2 8 48 40 26 29 13 46 24
#> [37,] 21 28 32 29 1 11 18 49 40 41 5 8 27 22 50
#> [38,] 5 1 41 18 23 28 8 12 22 40 47 49 25 20 29
#> [39,] 14 9 4 43 13 46 48 31 3 2 30 35 10 26 36
#> [40,] 8 29 1 28 50 18 27 41 12 21 25 32 24 5 36
#> [41,] 18 1 5 8 28 40 27 29 38 12 44 50 25 22 7
#> [42,] 9 58 94 39 46 14 99 13 2 26 61 4 31 35 10
#> [43,] 48 4 3 30 39 14 7 31 35 10 13 46 12 9 36
#> [44,] 27 18 41 24 22 1 8 28 5 40 32 12 29 25 7
#> [45,] 20 47 22 6 49 17 11 5 38 19 18 44 28 41 1
#> [46,] 2 13 31 35 26 4 10 3 39 30 48 36 9 14 43
#> [47,] 20 45 49 22 11 5 6 17 38 1 28 18 33 41 19
#> [48,] 3 30 4 43 31 35 10 7 36 39 12 50 13 46 2
#> [49,] 11 47 22 20 5 28 45 1 37 18 38 6 17 19 41
#> [50,] 8 36 40 29 12 27 3 25 30 24 1 35 41 18 10
#> [51,] 53 87 66 76 78 52 130 59 57 126 77 111 117 138 75
#> [52,] 57 87 66 76 92 86 53 78 138 128 71 51 75 111 62
#> [53,] 87 51 66 78 76 59 130 52 77 117 138 140 126 111 55
#> [54,] 81 90 82 70 91 63 93 94 80 120 99 95 58 69 88
#> [55,] 59 134 77 75 76 98 124 127 64 78 74 72 79 92 112
#> [56,] 100 97 95 83 79 96 74 68 89 91 67 72 64 93 65
#> [57,] 52 86 71 111 87 138 66 92 128 139 76 78 53 62 117
#> [58,] 94 99 82 81 61 54 90 70 60 80 91 107 95 93 83
#> [59,] 76 75 55 77 66 98 87 134 64 78 92 74 53 72 79
#> [60,] 95 91 90 100 65 56 83 70 107 97 99 93 85 89 80
#> [61,] 94 58 54 82 81 99 63 70 90 107 91 80 120 60 93
#> [62,] 79 92 67 97 89 64 96 98 139 100 72 56 65 128 85
#> [63,] 88 69 54 82 81 70 120 93 90 80 73 91 68 83 94
#> [64,] 79 92 98 74 134 75 62 72 55 56 97 128 84 127 139
#> [65,] 97 89 100 96 83 95 56 62 67 85 72 60 68 79 93
#> [66,] 87 76 53 52 51 59 78 75 77 57 55 92 98 64 117
#> [67,] 85 89 62 97 96 79 56 100 65 92 150 139 64 71 128
#> [68,] 83 93 80 95 100 56 91 70 72 74 97 90 65 96 98
#> [69,] 120 88 73 63 147 54 135 93 90 114 81 70 84 109 124
#> [70,] 81 90 82 93 80 91 83 54 68 95 100 60 99 56 63
#> [71,] 139 150 128 86 57 62 67 52 92 138 111 79 85 64 148
#> [72,] 98 74 83 75 79 64 100 97 56 68 92 62 93 55 95
#> [73,] 147 135 88 124 84 134 69 112 55 120 127 109 74 93 77
#> [74,] 64 98 72 79 75 56 134 92 100 68 97 83 55 84 59
#> [75,] 98 59 76 55 64 72 92 74 79 134 66 77 87 62 52
#> [76,] 66 59 87 75 55 53 78 52 98 77 92 64 51 134 79
#> [77,] 59 55 76 75 78 134 53 87 66 98 124 64 117 74 112
#> [78,] 87 117 53 148 138 76 66 113 55 77 59 104 111 140 52
#> [79,] 64 92 62 98 72 97 74 139 56 128 67 127 134 100 75
#> [80,] 70 93 68 83 82 81 90 95 91 100 65 99 72 56 60
#> [81,] 82 70 54 90 80 91 93 99 94 63 95 58 83 68 60
#> [82,] 81 70 54 90 80 91 93 99 94 58 63 68 83 95 60
#> [83,] 93 68 100 95 80 56 72 91 97 70 65 90 74 98 96
#> [84,] 134 127 124 102 143 135 79 64 73 74 112 56 104 147 55
#> [85,] 67 89 97 96 62 56 65 100 79 95 150 60 139 92 122
#> [86,] 57 71 52 62 92 139 111 128 150 67 138 149 87 66 96
#> [87,] 66 53 76 78 52 51 59 57 75 117 138 77 55 111 92
#> [88,] 69 73 120 63 93 147 54 135 70 81 90 84 82 83 124
#> [89,] 96 97 67 65 85 62 100 56 79 92 95 64 83 72 98
#> [90,] 70 91 81 93 54 82 95 80 60 83 68 100 56 99 107
#> [91,] 95 90 70 93 83 68 56 100 60 81 80 82 54 97 65
#> [92,] 64 79 98 62 75 74 139 128 72 96 97 52 134 76 67
#> [93,] 83 68 70 95 80 91 90 100 56 72 81 82 74 97 65
#> [94,] 58 99 82 61 81 54 70 90 80 60 91 107 93 63 95
#> [95,] 100 56 91 83 93 68 97 90 60 70 65 80 72 89 96
#> [96,] 89 97 67 65 62 100 56 85 92 79 64 98 72 95 83
#> [97,] 100 96 89 56 65 67 62 95 79 83 85 72 64 92 98
#> [98,] 75 64 72 92 74 79 62 59 55 76 134 97 56 100 96
#> [99,] 58 94 82 81 90 70 60 80 54 91 93 95 83 68 65
#> [100,] 56 97 95 83 65 68 89 96 72 93 79 91 62 67 74
#> [101,] 137 149 145 116 144 141 125 121 105 111 146 142 140 138 148
#> [102,] 143 122 84 114 127 124 115 150 112 139 147 128 79 104 134
#> [103,] 113 140 108 121 126 142 144 131 130 146 141 106 105 136 148
#> [104,] 117 129 138 148 128 112 127 134 133 150 124 139 105 78 84
#> [105,] 113 148 146 141 133 129 117 116 140 144 138 104 111 142 121
#> [106,] 136 123 108 131 103 126 130 119 140 121 113 144 142 141 146
#> [107,] 60 90 91 54 114 95 81 58 122 99 70 94 102 143 82
#> [108,] 131 130 103 106 123 126 136 113 140 78 117 53 119 121 144
#> [109,] 112 147 73 124 129 135 133 134 77 104 55 84 127 131 117
#> [110,] 118 145 121 144 125 132 137 101 126 141 142 140 149 116 103
#> [111,] 116 138 148 125 117 140 113 78 146 105 57 142 141 144 121
#> [112,] 124 129 127 133 147 104 134 84 109 55 148 117 102 143 73
#> [113,] 140 146 105 148 103 142 141 117 121 144 78 111 138 129 133
#> [114,] 102 143 122 147 84 115 124 127 73 112 135 120 107 90 95
#> [115,] 122 102 143 133 129 114 150 127 139 128 112 124 104 105 148
#> [116,] 111 141 137 149 101 125 145 105 146 144 148 121 142 140 138
#> [117,] 138 148 104 78 113 105 111 128 129 140 87 134 127 133 146
#> [118,] 132 110 126 125 121 145 144 106 136 140 103 137 142 101 141
#> [119,] 123 131 106 136 108 103 109 130 113 126 140 146 133 129 142
#> [120,] 69 88 73 63 54 147 135 114 90 81 84 70 93 91 82
#> [121,] 144 141 140 142 125 145 113 146 103 116 105 111 126 101 148
#> [122,] 102 143 115 150 114 139 84 127 128 67 85 79 124 56 62
#> [123,] 106 131 119 108 136 103 130 126 113 140 121 109 142 144 146
#> [124,] 127 112 84 134 147 55 73 104 129 102 143 133 128 135 79
#> [125,] 144 121 111 145 140 116 141 137 142 113 138 149 101 105 126
#> [126,] 130 103 140 108 121 125 144 113 106 53 142 51 78 131 117
#> [127,] 124 134 112 84 128 139 104 79 55 102 143 64 150 129 148
#> [128,] 139 150 127 104 71 79 138 92 148 64 117 62 134 111 124
#> [129,] 133 112 104 105 148 124 117 127 113 146 138 128 147 134 102
#> [130,] 108 126 103 131 53 78 140 113 51 117 87 77 106 138 148
#> [131,] 108 123 106 103 130 136 119 113 126 140 109 78 142 146 77
#> [132,] 118 110 126 106 125 136 121 144 145 140 103 130 51 142 108
#> [133,] 129 112 105 104 148 146 113 124 117 127 141 115 138 147 128
#> [134,] 55 64 127 84 124 74 104 79 59 75 98 112 135 92 77
#> [135,] 84 73 134 74 124 112 147 102 143 127 64 55 104 56 109
#> [136,] 106 123 103 131 108 126 119 121 140 130 142 113 144 141 146
#> [137,] 149 101 116 145 125 144 111 141 121 105 142 138 146 140 148
#> [138,] 117 148 111 104 78 128 105 113 139 150 87 116 57 125 140
#> [139,] 128 150 71 79 127 62 92 64 104 67 138 148 122 117 134
#> [140,] 113 142 121 103 146 144 141 125 148 111 105 78 117 126 116
#> [141,] 144 146 121 142 105 113 140 116 145 125 148 111 103 101 133
#> [142,] 140 146 141 121 113 144 103 116 111 148 125 145 105 78 117
#> [143,] 102 122 84 114 127 124 115 150 112 139 147 128 79 104 134
#> [144,] 121 141 125 145 140 142 113 116 105 146 103 101 111 137 148
#> [145,] 144 121 141 101 125 116 137 142 149 140 146 111 105 113 110
#> [146,] 113 142 141 105 140 148 116 121 144 133 111 103 129 117 125
#> [147,] 124 112 73 109 84 127 114 102 143 135 129 134 133 69 55
#> [148,] 117 138 113 105 146 111 78 104 129 140 128 133 116 141 142
#> [149,] 137 101 116 125 145 111 144 141 71 121 138 105 57 148 86
#> [150,] 139 128 71 79 104 127 67 92 62 122 64 138 102 143 84
#> [,16] [,17] [,18] [,19] [,20] [,21] [,22] [,23] [,24] [,25] [,26] [,27] [,28]
#> [1,] 49 22 44 11 24 7 23 47 20 36 45 3 30
#> [2,] 43 24 8 12 40 25 27 29 7 21 32 41 1
#> [3,] 39 8 26 27 40 14 24 29 9 41 1 18 28
#> [4,] 50 7 12 25 8 24 27 40 29 41 1 18 44
#> [5,] 27 44 25 45 37 7 21 50 32 24 17 6 36
#> [6,] 38 1 41 44 32 21 16 29 40 27 8 23 12
#> [7,] 44 5 36 4 24 29 31 28 35 10 22 46 39
#> [8,] 7 32 44 30 3 37 48 35 10 23 31 22 49
#> [9,] 50 7 12 25 24 8 27 42 40 29 41 1 18
#> [10,] 12 40 25 24 9 29 14 27 7 21 1 41 18
#> [11,] 21 29 32 40 8 44 15 27 33 12 25 23 50
#> [12,] 28 36 24 23 44 43 31 35 10 21 4 32 22
#> [13,] 50 12 24 8 25 7 40 27 29 21 41 1 18
#> [14,] 7 50 12 25 8 27 24 40 23 29 41 1 18
#> [15,] 1 38 18 21 32 41 29 44 40 8 27 23 12
#> [16,] 28 18 1 41 44 23 32 21 29 40 27 8 12
#> [17,] 38 1 41 44 32 21 29 16 23 40 27 8 12
#> [18,] 37 25 49 24 11 7 20 23 47 36 45 3 30
#> [19,] 32 18 1 41 38 29 44 40 27 8 24 16 12
#> [20,] 44 23 37 40 8 27 29 15 32 12 21 25 34
#> [21,] 36 12 25 44 38 22 35 10 30 3 47 7 19
#> [22,] 37 40 8 29 23 32 19 12 21 25 7 24 50
#> [23,] 47 50 29 49 3 48 45 43 30 11 36 24 37
#> [24,] 35 30 3 31 7 10 48 26 37 5 46 2 4
#> [25,] 48 24 44 36 31 23 21 35 10 43 4 32 22
#> [26,] 43 14 8 40 12 27 29 25 21 7 32 1 18
#> [27,] 21 36 38 30 3 22 37 48 35 31 23 10 49
#> [28,] 12 22 44 25 24 47 20 36 7 45 23 30 3
#> [29,] 25 38 44 49 11 7 22 3 30 35 10 31 48
#> [30,] 26 8 39 27 24 40 14 29 9 41 1 18 28
#> [31,] 12 24 25 14 8 7 27 40 29 41 1 21 18
#> [32,] 11 49 12 22 25 38 35 7 20 10 30 19 3
#> [33,] 1 23 18 41 37 44 29 40 8 21 32 27 12
#> [34,] 28 1 18 41 23 44 21 32 29 40 8 27 12
#> [35,] 8 12 40 25 27 9 29 7 14 21 41 1 32
#> [36,] 4 12 27 25 7 41 1 18 21 43 32 28 39
#> [37,] 38 19 24 44 47 20 12 36 25 45 17 6 7
#> [38,] 27 7 11 45 50 44 37 21 32 24 36 3 30
#> [39,] 7 50 12 25 8 24 27 40 29 41 23 1 18
#> [40,] 38 37 44 7 30 3 35 49 22 10 48 11 31
#> [41,] 32 23 24 21 49 37 11 36 20 47 45 3 30
#> [42,] 82 43 48 3 30 81 36 80 70 54 50 90 60
#> [43,] 25 2 50 8 26 27 23 40 24 41 29 1 18
#> [44,] 50 38 21 45 49 20 37 23 11 47 36 30 3
#> [45,] 33 27 37 23 40 8 29 32 25 21 12 15 34
#> [46,] 50 24 12 7 8 27 25 40 29 41 32 21 18
#> [47,] 23 37 44 40 8 29 27 12 15 21 25 32 34
#> [48,] 25 14 8 26 27 9 40 24 41 29 1 18 23
#> [49,] 29 21 40 32 8 44 27 12 23 15 33 25 50
#> [50,] 28 31 48 7 21 32 4 5 26 2 44 38 13
#> [51,] 140 55 148 113 125 92 98 142 103 134 64 128 108
#> [52,] 59 98 139 64 117 79 148 150 55 134 104 127 96
#> [53,] 148 75 57 113 134 92 104 103 125 98 142 128 64
#> [54,] 83 60 61 107 68 100 114 56 73 84 135 72 65
#> [55,] 84 87 73 66 104 117 128 53 139 148 135 62 147
#> [56,] 62 85 98 84 92 60 90 70 80 134 75 139 102
#> [57,] 150 51 148 64 125 79 116 75 98 149 59 104 67
#> [58,] 68 63 100 65 56 42 97 89 85 96 72 120 88
#> [59,] 52 51 127 117 124 128 104 62 138 84 139 148 112
#> [60,] 81 68 67 82 96 54 58 122 62 72 79 94 102
#> [61,] 69 42 88 95 68 83 100 114 56 73 65 97 135
#> [62,] 150 71 75 74 52 127 83 95 134 76 84 55 57
#> [63,] 95 61 135 99 72 58 74 100 56 147 84 60 114
#> [64,] 59 100 96 150 76 67 89 124 104 83 52 95 68
#> [65,] 80 98 91 92 64 74 90 70 75 139 84 150 128
#> [66,] 138 134 111 128 148 79 62 72 139 74 130 127 104
#> [67,] 95 98 122 72 74 83 60 84 127 102 143 68 91
#> [68,] 81 82 89 64 79 60 75 62 92 84 67 54 134
#> [69,] 82 91 83 112 95 102 143 80 134 55 68 72 74
#> [70,] 94 58 72 65 97 74 88 73 84 120 96 69 89
#> [71,] 116 149 117 104 89 127 96 122 97 98 87 78 76
#> [72,] 65 96 134 59 89 84 76 127 80 67 124 91 139
#> [73,] 72 102 143 64 114 79 75 98 83 59 56 95 104
#> [74,] 93 62 95 96 135 127 76 73 89 67 124 91 77
#> [75,] 78 127 97 128 53 124 56 100 96 139 83 84 68
#> [76,] 72 74 117 57 128 62 127 138 148 139 104 124 111
#> [77,] 127 130 73 104 51 72 92 148 79 84 109 52 135
#> [78,] 134 51 128 130 127 75 146 105 112 124 92 129 142
#> [79,] 150 84 96 89 55 83 95 124 65 85 59 76 102
#> [80,] 54 97 74 63 96 89 58 98 94 79 88 64 62
#> [81,] 100 56 88 61 120 107 69 65 72 97 74 73 84
#> [82,] 100 61 56 88 65 120 72 69 97 107 74 73 84
#> [83,] 79 89 60 64 81 62 82 84 67 75 92 85 54
#> [84,] 150 128 139 122 72 114 98 92 95 100 97 62 93
#> [85,] 71 64 128 83 91 98 72 74 102 143 84 68 93
#> [86,] 89 79 85 64 76 116 98 117 137 97 148 78 125
#> [87,] 148 98 128 64 134 130 140 104 113 139 79 127 62
#> [88,] 68 80 74 91 72 55 109 134 95 114 112 77 127
#> [89,] 68 74 60 139 150 91 93 71 75 128 80 84 122
#> [90,] 94 58 65 97 63 72 84 74 88 114 120 73 89
#> [91,] 74 72 84 89 96 99 79 107 67 85 135 64 102
#> [92,] 150 59 55 89 56 127 100 71 66 87 57 84 104
#> [93,] 54 60 98 79 84 64 73 96 88 89 63 135 62
#> [94,] 83 68 100 42 65 56 120 88 97 69 72 89 114
#> [95,] 74 79 67 81 85 84 64 62 82 98 54 92 102
#> [96,] 74 68 75 139 93 91 60 150 71 128 80 52 84
#> [97,] 68 74 93 91 60 75 139 84 80 150 128 134 90
#> [98,] 83 127 89 66 68 128 52 139 84 67 77 65 87
#> [99,] 107 61 100 56 97 63 89 96 72 85 67 42 74
#> [100,] 64 98 60 85 80 90 92 70 84 75 134 139 127
#> [101,] 113 117 133 71 129 110 103 104 128 150 139 57 115
#> [102,] 135 129 56 133 64 73 67 95 100 62 74 85 97
#> [103,] 125 117 78 129 123 111 133 138 145 53 116 104 87
#> [104,] 113 55 64 102 143 79 111 92 146 135 87 59 77
#> [105,] 103 125 145 78 101 112 128 127 150 139 137 149 124
#> [106,] 125 105 145 78 117 53 148 110 129 133 138 111 51
#> [107,] 56 100 93 85 83 97 65 67 84 80 68 89 120
#> [108,] 142 105 146 77 148 109 141 129 138 104 125 133 51
#> [109,] 108 88 113 69 78 120 148 102 143 105 103 114 59
#> [110,] 111 136 106 113 146 105 138 148 130 108 117 51 53
#> [111,] 128 149 52 87 71 137 145 139 104 53 101 150 66
#> [112,] 135 128 78 105 113 77 139 150 138 146 64 79 114
#> [113,] 125 116 104 126 130 145 108 112 53 87 131 128 127
#> [114,] 129 91 69 133 56 134 54 150 79 93 139 104 100
#> [115,] 84 147 116 146 71 117 79 138 141 134 67 111 113
#> [116,] 113 117 128 71 133 129 104 139 78 150 57 103 52
#> [117,] 112 139 53 150 55 124 116 76 125 52 103 141 59
#> [118,] 130 149 108 111 51 116 113 123 53 146 131 105 138
#> [119,] 121 141 105 144 112 117 148 78 77 104 147 125 145
#> [120,] 109 124 102 143 95 112 83 134 68 74 127 80 56
#> [121,] 137 138 117 149 78 110 130 136 133 108 106 129 53
#> [122,] 104 71 64 95 112 100 97 129 134 133 147 92 60
#> [123,] 141 105 78 117 129 53 133 77 148 125 112 104 51
#> [124,] 64 139 148 117 77 109 150 78 74 75 98 59 72
#> [125,] 146 148 117 103 78 57 53 87 110 104 52 130 51
#> [126,] 136 111 138 141 87 148 146 105 145 123 66 116 110
#> [127,] 117 92 147 133 98 122 78 62 75 74 73 72 138
#> [128,] 78 52 84 67 98 57 55 102 143 122 129 112 76
#> [129,] 143 78 141 115 109 84 139 150 140 116 103 111 55
#> [130,] 123 66 142 121 59 76 136 125 144 104 146 105 111
#> [131,] 117 121 105 53 129 144 148 133 141 112 104 138 87
#> [132,] 137 141 53 111 123 113 101 149 131 116 87 146 138
#> [133,] 102 143 140 78 116 109 139 150 142 134 103 84 111
#> [134,] 128 117 73 72 78 76 139 150 102 143 138 148 62
#> [135,] 79 120 88 114 93 98 72 91 95 77 92 75 59
#> [136,] 125 145 105 110 78 148 53 117 133 111 129 116 51
#> [137,] 71 113 57 110 117 86 128 150 139 104 103 133 52
#> [138,] 52 71 129 53 146 127 134 92 133 141 144 76 66
#> [139,] 84 102 143 124 98 52 111 57 85 97 55 78 75
#> [140,] 138 145 130 53 87 129 108 133 104 131 51 101 137
#> [141,] 137 117 138 129 149 78 104 126 128 112 108 130 139
#> [142,] 138 126 133 129 101 137 53 130 87 149 104 136 108
#> [143,] 135 129 56 133 64 73 67 95 100 62 74 85 97
#> [144,] 126 138 149 117 78 133 110 129 130 104 108 106 136
#> [145,] 103 148 138 126 117 133 129 78 104 57 128 136 71
#> [146,] 145 78 138 104 112 101 128 127 137 124 139 126 149
#> [147,] 120 88 104 122 115 77 79 64 128 148 74 117 72
#> [148,] 127 139 112 150 87 125 124 144 121 103 134 53 55
#> [149,] 142 146 140 128 117 150 113 139 52 104 78 110 133
#> [150,] 117 148 134 85 124 97 111 129 56 98 57 115 52
#> [,29] [,30]
#> [1,] 48 35
#> [2,] 18 28
#> [3,] 23 44
#> [4,] 28 23
#> [5,] 19 3
#> [6,] 25 24
#> [7,] 32 13
#> [8,] 4 11
#> [9,] 23 44
#> [10,] 28 32
#> [11,] 24 34
#> [12,] 37 49
#> [13,] 32 28
#> [14,] 38 44
#> [15,] 25 50
#> [16,] 25 24
#> [17,] 25 24
#> [18,] 48 35
#> [19,] 25 50
#> [20,] 7 50
#> [21,] 31 20
#> [22,] 33 15
#> [23,] 32 21
#> [24,] 38 22
#> [25,] 37 49
#> [26,] 41 28
#> [27,] 4 11
#> [28,] 19 35
#> [29,] 23 47
#> [30,] 44 21
#> [31,] 32 28
#> [32,] 45 47
#> [33,] 25 7
#> [34,] 25 7
#> [35,] 18 28
#> [36,] 44 37
#> [37,] 23 35
#> [38,] 48 17
#> [39,] 44 28
#> [40,] 23 4
#> [41,] 48 35
#> [42,] 24 7
#> [43,] 38 44
#> [44,] 48 35
#> [45,] 7 24
#> [46,] 1 44
#> [47,] 7 50
#> [48,] 38 44
#> [49,] 24 7
#> [50,] 46 37
#> [51,] 104 121
#> [52,] 77 72
#> [53,] 108 146
#> [54,] 97 102
#> [55,] 52 138
#> [56,] 143 150
#> [57,] 140 96
#> [58,] 67 69
#> [59,] 73 57
#> [60,] 143 84
#> [61,] 72 84
#> [62,] 59 86
#> [63,] 55 124
#> [64,] 135 87
#> [65,] 99 81
#> [66,] 140 86
#> [67,] 134 75
#> [68,] 135 85
#> [69,] 127 56
#> [70,] 107 79
#> [71,] 134 137
#> [72,] 73 128
#> [73,] 68 91
#> [74,] 65 128
#> [75,] 117 104
#> [76,] 150 112
#> [77,] 138 128
#> [78,] 139 57
#> [79,] 143 71
#> [80,] 75 67
#> [81,] 135 114
#> [82,] 96 89
#> [83,] 134 73
#> [84,] 83 129
#> [85,] 127 86
#> [86,] 75 53
#> [87,] 126 71
#> [88,] 56 100
#> [89,] 52 86
#> [90,] 69 102
#> [91,] 143 58
#> [92,] 138 78
#> [93,] 75 99
#> [94,] 96 85
#> [95,] 143 122
#> [96,] 134 76
#> [97,] 70 127
#> [98,] 95 93
#> [99,] 62 88
#> [100,] 81 102
#> [101,] 126 78
#> [102,] 92 55
#> [103,] 112 51
#> [104,] 140 71
#> [105,] 115 126
#> [106,] 109 116
#> [107,] 61 115
#> [108,] 87 112
#> [109,] 130 146
#> [110,] 57 78
#> [111,] 129 103
#> [112,] 115 59
#> [113,] 124 101
#> [114,] 60 128
#> [115,] 62 85
#> [116,] 115 87
#> [117,] 64 142
#> [118,] 57 87
#> [119,] 124 53
#> [120,] 72 55
#> [121,] 104 87
#> [122,] 89 135
#> [123,] 138 145
#> [124,] 114 122
#> [125,] 128 71
#> [126,] 104 52
#> [127,] 135 59
#> [128,] 75 87
#> [129,] 142 122
#> [130,] 55 141
#> [131,] 51 125
#> [132,] 57 78
#> [133,] 144 122
#> [134,] 56 129
#> [135,] 83 68
#> [136,] 138 109
#> [137,] 78 126
#> [138,] 64 142
#> [139,] 56 129
#> [140,] 136 128
#> [141,] 136 53
#> [142,] 128 131
#> [143,] 92 55
#> [144,] 53 87
#> [145,] 106 53
#> [146,] 87 53
#> [147,] 139 150
#> [148,] 52 76
#> [149,] 129 87
#> [150,] 112 89
#>
#> $distances
#> # A float32 matrix: 150x30
#> [,1] [,2] [,3] [,4] [,5]
#> [1,] 0.13119 0.13339 0.18709 0.23632 0.25927
#> [2,] 0.16559 0.17831 0.17831 0.23632 0.27029
#> [3,] 0.13339 0.16994 0.28294 0.31000 0.35186
#> [4,] 0.23632 0.24808 0.26539 0.28294 0.35186
#> [5,] 0.17831 0.25927 0.27029 0.29057 0.33791
#> [6,] 0.22659 0.44354 0.44844 0.46253 0.50591
#> [7,] 0.29729 0.39468 0.47724 0.49553 0.50375
#> [8,] 0.12076 0.23632 0.24808 0.24808 0.26539
#> [9,] 0.23632 0.33662 0.52163 0.55063 0.55063
#> [10,] 0.13119 0.18709 0.26539 0.27029 0.29604
#> # ...
#>
#>
#> $backend
#> [1] "cpu"
#>
#> $backend_used
#> [1] "native_cpu_exact"
#>
#> $method
#> [1] "native_exact"
#>
#> $metric
#> [1] "euclidean"
#>
#> $exact
#> [1] TRUE
#>
#> $exact_recall_by_construction
#> [1] TRUE
#>
#> $expected_recall_at_k
#> [1] 1
#>
#> $target_recall
#> [1] 0.99
#>
#> $target_met
#> [1] TRUE
#>
#> $recall_audited
#> [1] TRUE
#>
#> $recall_status
#> [1] "exact_by_construction"
#>
#> $tuning_policy
#> [1] "exact_below_5000"
#>
#> $tuning_rule
#> [1] "native_cpu_exact_n_lt_5000"
#>
#> $n_threads
#> [1] 2
#>
#> $input_type
#> [1] "numeric_to_float32"
#>
#> $distance_computation
#> [1] "float32"
#>
#> $cpu_fallback
#> [1] FALSE
#>
#> $timing
#> convert search
#> 0.000014875 0.001160708
#>
#> $n
#> [1] 150
#>
#> $k
#> [1] 30
#>
#> $exclude_self
#> [1] TRUE
#>
#> $backend_requested
#> [1] "cpu"
#>
#> $execution_backend
#> [1] "cpu"
#>
#> $engine
#> [1] "native_cpu_exact"
#>
#> $elapsed_sec
#> [1] 0.003
#>
#> $result_residency
#> [1] "host"
#>
#> attr(,"backend")
#> [1] "cpu"
#> attr(,"method")
#> [1] "native_exact"
#> attr(,"exclude_self")
#> [1] TRUE
#> attr(,"distance_type")
#> [1] "float32"
#> attr(,"class")
#> [1] "fastEmbedR_knn" "list"
#> attr(,"backend_requested")
#> [1] "cpu"
#> attr(,"metric")
#> [1] "euclidean"
#> attr(,"result_residency")
#> [1] "host"prepare_umap_knn() builds graph state and
epochs_per_sample once. prepare_tsne_knn()
validates, strips self-neighbors when needed, and stores the compact
affinity support. It uses one candidate neighbor per unit of perplexity
and trims wider KNN inputs. Prepared objects are useful for repeated
runs.
prepared_umap <- prepare_umap_knn(
knn,
graph_mode = "fuzzy",
n.cores = 2
)
prepared_tsne <- prepare_tsne_knn(knn, perplexity = 10)
layout_umap_reused <- umap_knn(
prepared_umap,
n.cores = 2,
seed = 4
)
layout_tsne_reused <- tsne_knn(
prepared_tsne,
Y_init = pca_fit$tsne_init,
n.cores = 2,
seed = 4
)
cbind(UMAP = dim(layout_umap_reused), TSNE = dim(layout_tsne_reused))
#> UMAP TSNE
#> [1,] 150 150
#> [2,] 2 2Independent UMAP initialization
umap_init() separates graph construction and sparse
initialization from layout optimization. The returned object retains the
exact prepared graph and initial coordinates, so umap_knn()
can reuse both without rebuilding either stage. This is useful for
repeated seeds, backend agreement tests, and optimizer-only timing.
umap_initialization <- umap_init(
prepared_umap,
backend = "cpu",
graph_mode = "fuzzy",
n.cores = 2,
seed = 4
)
layout_from_initialization <- umap_knn(
umap_initialization,
backend = "cpu",
n.cores = 2,
seed = 4
)
umap_initialization$timings
#> stage elapsed_sec
#> 1 knn 0.000
#> 2 graph 0.000
#> 3 initialization 0.005
#> 4 total 0.005The initialization object can also be passed to a Metal or CUDA optimizer. That controlled comparison holds the graph and starting coordinates fixed; ordinary one-call GPU embeddings continue to use their resident backend path.
UMAP graph modes
The graph mode is an explicit scientific choice, not a hidden speed setting.
-
graph_mode = "fuzzy"is the default and uses locally scaled fuzzy membership strengths from the standard UMAP graph construction. -
graph_mode = "binary"symmetrizes unit-weight neighbor edges and then uses the same UMAP optimizer. It is an explicit sensitivity mode and can produce stronger separation on some datasets.
layout_binary <- umap_knn(
knn,
graph_mode = "binary",
n.cores = 2,
seed = 4
)
layout_fuzzy <- umap_knn(
knn,
graph_mode = "fuzzy",
n.cores = 2,
seed = 4
)
old_par <- par(mfrow = c(1, 2), mar = c(1, 1, 2.5, 1), bg = "white")
plot_embedding(layout_binary, labels, "Binary graph")
plot_embedding(layout_fuzzy, labels, "Fuzzy graph")
UMAP graph modes use the same optimizer.
par(old_par)Record the chosen mode whenever embeddings are reported or compared.
Float32 and memory-conscious analysis
All native embedding buffers use float32 and integer indices. A
standard R numeric matrix is converted at the native boundary. Supplying
a float::float32 matrix avoids retaining an unnecessary
double-precision copy in R, and the returned layout remains float32.
if (requireNamespace("float", quietly = TRUE)) {
x_float <- float::fl(x)
fit_float <- umap(
x_float,
n_neighbors = 15,
graph_mode = "fuzzy",
standardize = FALSE,
n.cores = 2,
seed = 4
)
c(
input_class = class(x_float)[1],
layout_class = class(fit_float$layout)[1]
)
}
#> input_class layout_class
#> "float32" "float32"For large datasets:
- perform scaling or feature extraction once and reuse the result;
- use float32 input when practical;
- leave
keep_knn = FALSEunless neighbor matrices are needed later; - use prepared KNN/graph objects for repeated seeds;
- consider landmarking only when its approximation is scientifically acceptable.
Embedding new observations
transform_tsne() places query observations into an
existing reference embedding without moving the reference points. It
builds query-to-reference affinities, initializes each query near its
reference neighbors, and optimizes only the query coordinates. Reference
and query data must have undergone the same preprocessing.
For this executable example, every fifth Iris observation is treated
as new. We compute its exact query-to-reference KNN directly. The same
API accepts any plain host list with query-to-reference
indices and distances.
query_rows <- seq(5L, nrow(x), by = 5L)
reference_rows <- setdiff(seq_len(nrow(x)), query_rows)
reference_data <- x[reference_rows, , drop = FALSE]
query_data <- x[query_rows, , drop = FALSE]
reference_fit <- tsne(
reference_data,
perplexity = 10,
standardize = FALSE,
n.cores = 2,
seed = 4
)
exact_query_knn <- function(reference, query, k) {
squared_distances <- outer(
rowSums(query^2), rowSums(reference^2), "+"
) - 2 * tcrossprod(query, reference)
squared_distances[squared_distances < 0] <- 0
indices <- t(apply(squared_distances, 1L, order))[
, seq_len(k),
drop = FALSE
]
distances <- matrix(
sqrt(squared_distances[
cbind(rep(seq_len(nrow(query)), each = k), as.vector(t(indices)))
]),
nrow = nrow(query),
byrow = TRUE
)
list(indices = indices, distances = distances)
}
query_knn <- exact_query_knn(reference_data, query_data, k = 15)
query_layout <- transform_tsne(
reference_fit,
knn = query_knn,
perplexity = 5,
n_iter = 100,
n.cores = 2,
backend = "cpu",
seed = 4
)
reference_labels <- factor(labels[reference_rows], levels = levels(labels))
query_labels <- factor(labels[query_rows], levels = levels(labels))
par(bg = "white")
plot(
reference_fit$layout,
col = embedding_colors[as.integer(reference_labels)],
pch = 19,
cex = 0.65,
asp = 1,
axes = FALSE,
xlab = "",
ylab = "",
main = "Fixed-reference transform"
)
points(
query_layout,
col = embedding_colors[as.integer(query_labels)],
pch = 17,
cex = 1.05
)
legend(
"bottomright",
legend = c("Reference", "New"),
pch = c(19, 17),
bty = "n",
cex = 0.8
)
box(col = "#B8B8B8")
Transform into a fixed t-SNE reference.
For Metal or CUDA, request the same backend explicitly. If the native fixed-reference path is unavailable, the call fails rather than relabeling a CPU transform as GPU work.
Landmarking
Landmarking embeds a representative subset and places the remaining
rows in that reference. It is never activated silently by
umap() or tsne(). The final layout still has
one row per input observation. Reference and projection searches use
fastEmbedR’s native KNN routes.
The one-call functions perform selection, reference fitting, query-to-reference neighbor search, projection, and optional fixed-reference refinement.
fit_landmark_umap <- umap(
x,
landmarks = 0.5,
n_neighbors = 15,
graph_mode = "fuzzy",
standardize = FALSE,
n.cores = 2,
seed = 4
)
fit_landmark_tsne <- tsne(
x,
landmarks = 0.5,
perplexity = 10,
standardize = FALSE,
n.cores = 2,
seed = 4
)
rbind(
UMAP = fit_landmark_umap$metrics[
1, c(
"elapsed", "reference_embedding_elapsed",
"landmark_projection_knn_elapsed", "landmark_refinement_elapsed"
)
],
TSNE = setNames(
fit_landmark_tsne$metrics[
1, c(
"elapsed", "reference_embedding_elapsed",
"landmark_projection_knn_elapsed", "transform_elapsed"
)
],
c(
"elapsed", "reference_embedding_elapsed",
"landmark_projection_knn_elapsed", "landmark_refinement_elapsed"
)
)
)
#> elapsed reference_embedding_elapsed landmark_projection_knn_elapsed
#> UMAP 0.198 0.194 0.001
#> TSNE 14.480 14.472 0.002
#> landmark_refinement_elapsed
#> UMAP 0.003
#> TSNE 0.006
old_par <- par(mfrow = c(1, 2), mar = c(1, 1, 2.5, 1), bg = "white")
plot_embedding(fit_landmark_tsne$layout, labels, "Landmark t-SNE")
plot_embedding(fit_landmark_umap$layout, labels, "Landmark UMAP")
Iris with 50% landmarks.
par(old_par)CPU uses exhaustive exact query-to-reference search below 5,000 reference rows and native HNSW otherwise. Metal uses native exact search for small references and recall-tuned IVF-Flat for larger references, followed by fixed-reference transforms. CUDA uses native exact or IVF-Flat query search and keeps KNN, layouts, and refinement buffers resident on the GPU. The result records landmark count, fraction, projection backend, and separate reference, query-KNN, and transform timings so a landmark approximation cannot be mistaken for a full embedding.
Landmarking can also reduce the memory needed by reference
optimization. The returned metrics include
landmark_reference_memory_saving for this stage and
landmark_projection_knn_bytes for the query search. The
estimates use float32 graph values and coordinates and exclude the input
matrix, R object overhead, allocator caches, and temporary buffers. They
describe where memory is saved rather than claiming that every landmark
run has lower peak RSS. Projection may retain enough query neighbors or
output rows to offset the reference-stage saving.
Each integrated landmark fit also retains a reusable model in
$model.
project_landmark_model(fit$model, newdata) applies the
fitted preprocessing transform and projects genuinely new observations
while keeping the reference coordinates fixed.
Measuring embedding quality
evaluate_embedding() reports local, global, and
label-aware metrics. For large datasets it uses deterministic subsamples
rather than constructing a complete all-pairs distance matrix. Here all
Iris observations are small enough to evaluate directly. Supplying
reference_nn also avoids repeating the high-dimensional
neighbor search.
quality_umap <- evaluate_embedding(
x,
fit_umap$layout,
labels = labels,
k = 10,
primary_k = 10,
reference_nn = knn,
sample_size_for_local_metrics = nrow(x),
sample_size_for_global_metrics = nrow(x),
seed = 4,
method = "UMAP",
backend = "cpu",
dataset = "iris",
n.cores = 2
)
quality_tsne <- evaluate_embedding(
x,
fit_tsne$layout,
labels = labels,
k = 10,
primary_k = 10,
reference_nn = knn,
sample_size_for_local_metrics = nrow(x),
sample_size_for_global_metrics = nrow(x),
seed = 4,
method = "t-SNE",
backend = "cpu",
dataset = "iris",
n.cores = 2
)
quality <- rbind(quality_umap, quality_tsne)
quality[, c(
"method", "trustworthiness", "knn_preservation_10",
"distance_spearman", "silhouette", "label_knn_accuracy"
)]
#> method trustworthiness knn_preservation_10 distance_spearman silhouette
#> 1 UMAP 0.9745576 0.7226667 0.8969430 0.5223689
#> 2 t-SNE 0.9844511 0.7953333 0.9371874 0.4401343
#> label_knn_accuracy
#> 1 0.9533333
#> 2 0.9600000No single score is sufficient. Trustworthiness and neighbor preservation measure local structure; distance correlation is a coarse global diagnostic; silhouette and KNN label accuracy are meaningful only when labels are valid external annotations rather than tuning targets.
CPU, Metal, and CUDA
The backend argument has exactly three choices.
| Backend | One-call KNN | Embedding implementation |
|---|---|---|
"cpu" |
Exact below 5,000 rows; HNSW otherwise | Multithreaded C++ |
"metal" |
Exact or IVF-Flat | Metal kernels |
"cuda" |
Exact or IVF-Flat | CUDA kernels |
CPU t-SNE uses a fast Fourier transform (FFT) grid. Metal uses native Objective-C++ kernels. CUDA uses RAPIDS cuVS search and keeps embedding buffers resident on the device.
GPU use is explicit:
fit_metal <- tsne(x, perplexity = 10, backend = "metal", seed = 4)
fit_cuda <- umap(x, n_neighbors = 15, backend = "cuda", seed = 4)If the requested backend was not compiled or is unavailable at runtime, these calls stop. CPU fallback is not relabeled as Metal or CUDA. Because parallel reductions and atomic updates can be ordered differently, fixed seeds provide controlled stochastic initialization and sampling but do not promise bitwise identity between devices or thread counts.
Package boundary
The package owns the matrix and KNN-input interfaces, native
nearest-neighbor routes, PCA initialization, graph or affinity
construction, low-dimensional optimization, landmark transformation,
graph construction, community detection, quality metrics, and returned
metadata. A plain host KNN object can be supplied as
indices and distances, while a CUDA KNN result
produced by precompute_knn() can remain device-resident
through the CUDA embedding path. No external R nearest-neighbor package
is called by the production workflow.
UMAP implementation
The fuzzy path follows the graph construction and cross-entropy
optimization defined by McInnes, Healy, and Melville.
fastEmbedR changes the execution boundary rather than the
UMAP objective: graph weights, coordinates, schedules, and optimizer
state use compact float32 buffers, and dense graph intermediates are
avoided. CPU parallelizes row preparation and sampled-edge updates;
Metal retains graph and optimizer buffers in unified memory; CUDA
constructs and optimizes the graph from device-resident KNN arrays.
graph_mode = "fuzzy" is the standard default. The
explicit "binary" mode assigns unit weight to the symmetric
KNN union and is an adjacency-only sensitivity analysis for settings
where neighbor identity is more reliable than distance calibration. It
is not standard UMAP and is not guaranteed to be faster because every
retained edge receives the maximum sampling frequency.
Interpolation-based t-SNE implementation
The affinity definition and Kullback–Leibler objective follow van der
Maaten and Hinton. Interpolation-based repulsion follows FIt-SNE, while
the automatic two-phase optimization policy is informed by opt-SNE and
related t-SNE implementations. The compact policy uses
ceiling(perplexity) non-self affinity candidates.
CPU uses a retained FFT-grid workspace, Metal uses package-native
scatter, transform, gather, attraction, and optimizer kernels, and CUDA
combines native kernels with cuFFT and persistent device buffers. PCA
initialization is created by pca(..., tsne_init = TRUE) or
internally by the one-call workflow. The fixed-reference transform
follows the published reference-space embedding procedure and keeps the
fitted reference coordinates unchanged.
Graph clustering and landmarking
knn_graph() builds weighted sparse graphs using
shared-neighbor, inverse- distance, or binary edges.
graph_cluster() provides native Louvain, Leiden, and exact
CPU Walktrap implementations. Louvain and Leiden have CPU, Metal, and
CUDA routes; Walktrap remains CPU-only because its exact transition
representation is quadratic in the number of vertices. The
implementations are tested against igraph on matched
weighted graphs, but igraph is not a runtime
dependency.
Landmark workflows separate selection, reference embedding, projection KNN, local correction, and optional fixed-reference refinement. They are explicit approximations: timings and returned parameters identify landmark and projection stages rather than presenting them as ordinary full-data embeddings.
Backend reporting and software acknowledgements
An explicit accelerator request must execute on that backend or fail. Returned objects record the resolved embedding backend, nearest-neighbor route, precision, parameters, and stage timings. An unavailable explicit accelerator request raises an error rather than running on CPU.
The package is distributed under the MIT license. Adapted source and
linked library boundaries, upstream commits, SPDX identifiers, and
retained notices are recorded in inst/NOTICE,
inst/COPYRIGHTS,
inst/THIRD_PARTY_DEPENDENCIES.json, and the repository’s
source-provenance documentation. Software used only for validation
remains outside the production call graph.
A practical analysis checklist
Before publishing an embedding analysis:
- state whether input features were centered, scaled, or reduced by PCA;
- report method, backend, seed,
n_neighborsor perplexity, and UMAP graph mode; - distinguish full runs from landmark approximations;
- compare total runtime when competing packages expose different internal boundaries;
- report trustworthiness and neighbor preservation alongside any label-aware score;
- inspect the plot for outliers, splitting, crowding, and seed sensitivity;
- save
sessionInfo()and the result’s$parametersand$timings.
References
- van der Maaten L, Hinton G. Visualizing data using t-SNE. J Mach Learn Res 2008;9:2579-2605.
- van der Maaten L. Accelerating t-SNE using tree-based algorithms. J Mach Learn Res 2014;15:3221-3245.
- Linderman GC, Rachh M, Hoskins JG, Steinerberger S, Kluger Y. Fast interpolation-based t-SNE for improved visualization of single-cell RNA-seq data. Nat Methods 2019;16:243-245.
- Policar PG, Strazar M, Zupan B. openTSNE: a modular Python library for t-SNE dimensionality reduction and embedding. J Stat Softw 2024;109:1-30.
- Chan DM, Rao R, Huang F, Canny JF. t-SNE-CUDA: GPU-accelerated t-SNE and its applications to modern data. arXiv 2018;1807.11824.
- Belkina AC, Ciccolella CO, Anno R, Halpert R, Spidlen J, Snyder-Cappione JE. Automated optimized parameters for T-distributed stochastic neighbor embedding improve visualization and analysis of large datasets. Nat Commun 2019;10:5415.
- Policar PG, Strazar M, Zupan B. Embedding to reference t-SNE space addresses batch effects in single-cell classification. Mach Learn 2021;110:1211-1231.
- McInnes L, Healy J, Melville J. UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. arXiv 2018;1802.03426.
- Johnson J, Douze M, Jegou H. Billion-scale similarity search with GPUs. IEEE Trans Big Data 2021;7:535-547.
- Blondel VD, Guillaume JL, Lambiotte R, Lefebvre E. Fast unfolding of communities in large networks. J Stat Mech 2008;2008:P10008.
- Traag VA, Waltman L, van Eck NJ. From Louvain to Leiden: guaranteeing well-connected communities. Sci Rep 2019;9:5233.
- Pons P, Latapy M. Computing communities in large networks using random walks. J Graph Algorithms Appl 2006;10:191-218.
sessionInfo()
#> R version 4.6.1 (2026-06-24)
#> Platform: aarch64-apple-darwin23
#> Running under: macOS Tahoe 26.6.2
#>
#> Matrix products: default
#> BLAS: /Library/Frameworks/R.framework/Versions/4.6/Resources/lib/libRblas.0.dylib
#> LAPACK: /Library/Frameworks/R.framework/Versions/4.6/Resources/lib/libRlapack.dylib; LAPACK version 3.12.1
#>
#> locale:
#> [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
#>
#> time zone: UTC
#> tzcode source: internal
#>
#> attached base packages:
#> [1] stats graphics grDevices utils datasets methods base
#>
#> other attached packages:
#> [1] fastEmbedR_0.1
#>
#> loaded via a namespace (and not attached):
#> [1] cli_3.6.6 knitr_1.52 rlang_1.3.0 xfun_0.61
#> [5] otel_0.2.0 textshaping_1.0.5 jsonlite_2.0.0 htmltools_0.5.9
#> [9] ragg_1.5.2 sass_0.4.10 rmarkdown_2.32 evaluate_1.0.5
#> [13] jquerylib_0.1.4 fastmap_1.2.0 yaml_2.3.12 lifecycle_1.0.5
#> [17] RhpcBLASctl_0.23-42 compiler_4.6.1 fs_2.1.0 Rcpp_1.1.2
#> [21] systemfonts_1.3.2 float_0.3-4 digest_0.6.39 R6_2.6.1
#> [25] parallel_4.6.1 bslib_0.12.0 withr_3.0.3 tools_4.6.1
#> [29] pkgdown_2.2.1 cachem_1.1.0 desc_1.4.3