Single neurons in the human medial temporal lobe flexibly shift representations across spatial and memory tasks

Investigations into how individual neurons encode behavioral variables of interest have revealed specific representations in single neurons, such as place and object cells, as well as a wide range of cells with conjunctive encodings or mixed selectivity. However, as most experiments examine neural activity within individual tasks, it is currently unclear if and how neural representations change across different task contexts. Within this discussion, the medial temporal lobe is particularly salient, as it is known to be important for multiple behaviors including spatial navigation and memory, however the relationship between these functions is currently unclear. Here, to investigate how representations in single neurons vary across different task contexts in the MTL, we collected and analyzed single-neuron activity from human participants as they completed a paired-task session consisting of a passive-viewing visual working memory and a spatial navigation and memory task. Five patients contributed 22 paired-task sessions, which were spike sorted together to allow for the same putative single neurons to be compared between the different tasks. Within each task, we replicated concept-related activations in the working memory task, as well as target-location and serial-position responsive cells in the navigation task. When comparing neuronal activity between tasks, we first established that a significant number of neurons maintained the same kind of representation, responding to stimuli presentations across tasks. Further, we found cells that changed the nature of their representation across tasks, including a significant number of cells that were stimulus responsive in the working memory task that responded to serial position in the spatial task. Overall, our results support a flexible encoding of multiple, distinct aspects of different tasks by single neurons in the human MTL, whereby some individual neurons change the nature of their feature coding between task contexts.

Introduction 116 task contexts (Behrens et al., 2018;Duncan, 2001). Overall, there is thought to be a hierarchy of encoding 117 from specialized neurons in primary sensory areas that are not expected to change their representation, 118 to higher-order areas that encode more abstract features, with more flexibility to encode multiple 119 features, both simultaneously and/or across time. Further understanding the consistency of 120 representations, however, requires dedicated work that evaluates neural activity across task contexts.

122
In the cortex, some experiments have examined different task variants within a cognitive domain, 123 to investigate how individual neurons change their activity in relation to different task demands. For 124 example, in recordings in the parietal cortex from mice, neural responses across two different visual 125 decision tasks engaged largely distinct populations of neurons (Lee et al., 2022). However, when using 126 two different categorization tasks in monkeys, a broadly similar neural representation was found across 127 the two tasks (Mohan et al., 2021). An early experiment in humans did report a small number of MTL 128 neurons that showed responses to words and also to unrelated faces (Heit et al., 1988). Some studies in 129 humans have examined responses to the same stimuli based on task demands, for example, finding 130 differences in responses across regions to different task demands when viewing the same faces (Cao, Within the discussion of neural representations, the medial temporal lobe (MTL) is a structure of 138 particular interest due to its involvement in multiple cognitive processes. In spatial navigation, the MTL 139 has seemingly 'specialist' cells that encode specific locations in space (place cells), as well as location-and 140 navigation-related features such as head direction, speed, and environment borders (Moser et al., 2017).

141
While much of the spatial navigation literature is in rodents, space-related representations have also been 142 found in non-human primates (Rolls & Wirth, 2018), as well as recent demonstrations of place, target-143 location, and sequence encodings in single-neurons in humans (Miller et al., 2013;Tsitsiklis et al., 2020).

144
In investigations with visually presented stimuli, the MTL has also been found to have neurons that 145 respond to broadly tuned object categories as well as to highly specific concepts, which is also thought to 146 relate to memory processes (Quiroga,

152
The previous findings, whereby the MTL has been found to engage in seemingly distinct functions 153 across spatial navigation and memory has led to the suggestion that this structure may engage in 154 representing 'cognitive maps' whereby individual neurons represent features or relations within a high-155 dimensional state space (Behrens et al., 2018;Schiller et al., 2015;Tolman, 1948). Under this hypothesis, 156 the MTL constructs 'maps' of features of interest within which physical space may simply be a special case 157 of a more general mechanism that can be applied to other 'spaces'. In a physical context, the activity of 158 place cells represent locations in the map, representing physical space. In a different context, individual 159 MTL neurons are predicted to be able to represent elements within other feature spaces, such that they 160 can flexibly engage in different kinds of representations based on task demands. This perspective is 161 supported by studies such as one that finds 'frequency-place cells' wherein individual neurons represent 162 locations in frequency space during a sound modulation task (Aronov et al., 2017). This framework is also 163 consistent with perspectives whereby the hippocampus can be thought of as a general relational 164 processing system which can be applied to organizing relations across space, time, and conceptual 165 dimensions (Eichenbaum & Cohen, 2014).

167
Empirically, the specificity, flexibility, and consistency of neural responses can be examined by  tasks. Such experimental designs are therefore a situation in which human participants may be ideal due 176 to their capacity to rapidly learn and switch between distinct behavioral contexts.

178
Collectively then, the existing evidence suggests that the brain appears to use a hierarchically 179 organized combination of 'specialist' and 'generalist' cells.

217
Patients had between 3-6 Behnke-Fried electrodes implanted, to a total of 22 electrodes across the group, 218 each of which had 8 micro-wires. Recordings from the microwires were collected at 32 kHz using a 219 NeuraLynx Atlas recording system (Neuralynx, Bozeman, USA) with full bandwidth recordings (0.1-9 kHZ).

220
The data analyzed in this study is openly available for the one-back task  and will be 221 released for the Treasure Hunt task upon publication.

225
Participants in our study performed a paired task session in which they completed a visual working 226 memory task, which was immediately followed by a spatial navigation and episodic memory task. The 227 visual working memory task was a one-back paradigm, which is commonly used to test how participants 228 maintain and manipulate information in working memory (Cao et al., 2021;. The 229 spatial navigation task is a 3D virtual stimulus-location associative memory task called Treasure Hunt (TH), 230 developed in Unity, which was previously used to study various aspects of human spatial memory and 231 electrophysiology (Miller et al., 2018;Tsitsiklis et al., 2020). Both tasks were played on bedside laptops, 232 with participants pressing the spacebar on a keyboard to respond in the one-back and using a separate 233 joystick to control movements in the Treasure Hunt task. In each paired-task session, participants typically 234 started with the one-back task first and played Treasure Hunt immediately after, with short breaks 235 between tasks if necessary.

237
Each of these tasks could occur in one of two versions, a "face" version where presented items 238 were famous faces, or an "object" version where items were images of general objects. Within a single 239 paired task session, the stimuli type was always consistent across tasks (eg. the object version of the one-240 back task was always followed by the object version of TH). In the "object" version of the one-back task, 241 10 images of objects from 50 categories were taken from the ImageNet database (Deng et al., 2009). This

250
We used a one-back task with human faces or objects as described in previous studies (Cao et al.,

261
The Treasure Hunt spatial navigation and memory task was used, as described in previous studies 262 (Miller et al., 2018;Tsitsiklis et al., 2020). In each trial of Treasure Hunt, participants use a joystick to 263 navigate a rectangular arena on a virtual beach and encounter treasure chests that can contain items.

264
Participants are instructed to remember the location of the presented items, so that they can later report

272
Each trial of Treasure Hunt consists of two phases -a navigation / encoding phase, and a retrieval 273 phase. During the navigation phase, participants are first placed at one end of the arena, from which they 274 can navigate freely using the joystick. They are instructed to navigate to a series of chests that are 275 presented serially in the arena. Upon reaching a chest, players are rotated to the front of the chest, at 276 which point it opens and reveals either an item contained in the chest, which is presented for 1.5 seconds, 277 or the chest is shown to be empty. There are 4 chest presentations per trial, 2 or 3 of which are full chests.

278
After reaching all four chests of a trial, participants are transported to one end of the arena, either the 279 same side as the navigation start or the opposite side, indicating the end of the navigation phase.

280
Participants then play a distractor task which is a computerized version of the "shell game". After the 281 distractor game, the recall phase starts in which participants are prompted with each of the items from 282 the trial in a random order. They are first asked to rate their confidence level of whether they remember 283 the location of the encountered item (response options: "Yes", "Maybe" or "No''). They are then asked to 284 respond with the exact location in the arena they encountered the item by maneuvering a crosshair with 285 the joystick. At the end of the recall period, participants receive feedback regarding whether each 286 response is close enough to be considered correct, and receive points accordingly. A response is 287 considered correct if it is within 13 virtual units of the true object location.

294
After data collection, each paired-task session was pre-processed together such that the same 295 putative single-neurons could be isolated and analyzed across both tasks. Single-neuron activity was 296 identified and spike sorted using the OSort algorithm (Rutishauser et al., 2006). Spike times were then 297 extracted from the full session for each task, such that each task could be analyzed independently. Only 298 neurons with an average firing rate greater than 0.15 Hz across both tasks were kept for analysis, keeping 299 a total of 1173 neurons across the project (face-version: 608 neurons; object-version: 565 neurons). In 300 the analysis, neurons from each recording session were considered unique, even if they came from the 301 same subject. We note that the same participants did both stimulus versions of the paired-task sessions, 302 such that there could be overlapping neurons in the two datasets. Notably, the two stimulus versions 303 were analyzed independently. For the one-back task, single-neuron activity was associated with the 304 behavioral timestamps and analyzed using custom scripts in the Matlab programming language 305 (Mathworks, Inc, USA). For the Treasure Hunt task, single-neuron activity was organized together with 306 behavioral information into Neurodata Without Borders (NWB) files (Rübel et al., 2022) which were then 307 analyzed in the Python programming language, using the spiketools module for analyzing single-neuron 308 activity (Donoghue et al., 2023). After each task was analyzed, the activity patterns and task activations 309 within each task were then compared across tasks in order to examine single-neuron activity across task 310 contexts.

314
Across both tasks, we analyzed neural responses that were potentially consistent across both 315 tasks, such as responses to stimuli, and also responses that were specific to each task, for example,

328
In the one-back task, we analyzed neurons for non-selective responses to stimuli, as well as for 329 selective responses to specific stimuli, following previously described procedures (Cao et

340
After detecting neurons with responses to particular identities or categories, we employed a 341 series of control analyses to further examine these neurons. We assessed the selectivity of each neuron 342 to different identities for each neuron using an identity selectivity index defined as the dʹ between the 343 most-and least-preferred identities (Grossman et al., 2019). This was computed as (μbest -μworst / 344 sqrt(0.5*(σ 2 best + σ 2 worst))), wherein μbest, μworst and σ 2 best, σ 2 worst denote the mean firing and variance of 345 firing rate for the most-and least-preferred identities, respectively. We also computed a depth of to the least preferred, such that the response ratio of the most preferred identity is always 1. We 354 compared response ratio for each ordered identity between identity vs. non-identity-selective neurons 355 using two-tailed unpaired t-test (corrected for multiple comparisons using false discovery rate (FDR) 356 (Benjamini & Hochberg, 1995

367
To examine potential space-and sequence-related neural responses in TH, we analyzed the data 368 from the navigation periods. For the space related analyses, we first binned the rectangular environment 369 into a grid such that we could assign the position of the subject, the position of the targets (chests) and

400
To visualize potential relationships between responses to different features, we did a series of 401

417
In this project, we used a paired-task session (Fig 1A), in which 5 neurosurgical patients completed 418 22 paired task sessions consisting of a working memory one-back (OB) task followed by the Treasure Hunt

419
(TH) spatial navigation task (Fig 1A&B). There were two versions of these paired-task sessions, one using 420 face stimuli and the other using object stimuli. We recorded from implanted microwires during the paired 421 task session, that were pre-processed together such that the same single-neurons could be compared 422 across both tasks (Table 1;  that have task specific responses, neurons that maintain a representation across task contexts, and 428 neurons that switch representations between tasks.

432
Behaviorally, participants performed consistently well at the n-back task, both in terms of 433 accuracy of repetition detection (Fig 1C; OB-

437
sample t-test) between the face and object versions of the one-back task. Analyzing the neural data, we 438 first examined whether there were stimulus reactive cells with a significant activation for presented 439 images, using paired t-tests ( Fig 2A&D). We detected a significant number of stimulus-responsive cells

443
We subsequently analyzed neural responses for cells that responded selectively to individual 444 identities (people or objects) using 1-way ANOVA (see methods for details; Fig 2B&E). We found a

457
In the Treasure Hunt task, we measured performance on each trial based on the distance between 458 the subject's response location to the item's actual position. Participants responded accurately on ~33% 459 of trials (Fig 1C; TH- responsive cells during the chest-opening events (Fig 3A&D), finding a significant number of stimulus-463 responsive cells (Fig 3G; (Fig 3B&E), we did find a significant number of spatial target cells (Fig 3H; TH-face:

482
We next examined the overlap between neuronal responses across the two tasks, starting by 483 comparing stimulus responses across both tasks. To do so, we used Chi-squared tests to examine the 484 number of cells that responded across one or both tasks. We found a significant overlap between the 485 neurons that were found to be stimulus-responsive in one-back and those that were stimulus-responsive 486 in Treasure Hunt (Fig 4A-C Table 2. Examples of neurons with responses across both task contexts, for neurons with different representations, responding to different aspects of each task for face stimuli (A-C) and for object stimuli (D-F). A&D) Example neurons that respond to stimuli in the one-back task and to serial position in the Treasure Hunt task. B&E) Average waveforms for the example neurons in A&D, split across tasks. The waveforms are very similar, motivating that these detected units are well isolated and represent the same neuron across tasks. C&F) Shows the number of task responsive neurons, including the number of neurons that respond in only one task, and the number of overlap neurons that respond in both tasks.  responses, having uncorrelated responses, having task specific responses, and having a combination of 516 task specific responses and neurons that respond to both tasks (Fig 6A-D). The empirical distributions 517 appeared qualitatively consistent with the simulated data that was created with a combination of task-518 specific and overlapping responses (Fig 6E-F). Specifically, in the empirical data, there were no significant 519 correlations between the statistical measures (spearman correlation, all p's > 0.05), which is also 520 consistent with their being a combination of task specific and task general responses. Overall, we conclude 521 from these analyses that there are neurons that do respond across tasks, but that this appears to 522 represent a subset of neurons, as many neurons appear to respond in a task-specific manner.

Figure 6 -Group level comparison of responses across tasks across all neurons. A-D) Simulated data
showing hypotheses of potential relationships of the task-related activity across two distinct tasks. Four potential relationships are shown: A) responses are correlated between tasks, B) responses are uncorrelated across tasks, C) responses are task-specific, such that individual neurons respond to single task only, D) there is a combination of task-specific responses, whereby most neurons respond to one task only, with a subset of neurons that respond to both ('overlap' neurons). E-F) Empirical distributions comparing the statistical measures for different responses between tasks, shown for the pairings in which we saw evidence of a significant number of overlap neurons. Each data point represents an individual neuron, plotted based on the statistical measures computed for different analyses (f-value or t-value, depending on the analysis -note that t-values are absolute valued). Each data point is colored by the outcomes of the analyses (yellow: not significant in either task; purple: only significant in the one-back task measure; green: only significant in the Treasure Hunt task measure; black: significant in both the one-back and Treasure Hunt task measures. Inset text shows the correlation values. The overall pattern of the empirical data is most consistent with the simulated hypothesis in which there is a combination of task specific and overlap neurons.

527
In this study, we leveraged the human capacity for rapidly shifting task demands to investigate 528 representations in individual neurons in the MTL across distinct behavioral contexts. We did so using a 529 paired-task session including a one-back visual working-memory task followed by a spatial-navigation and 530 memory task with human participants. Of the neurons that responded in both tasks, some neurons do 531 maintain a consistent representation, by responding to the same class of stimuli across task contexts, 532 whereas other neurons appear to switch their representation, responding to one type of feature in one 533 task, and switching to represent a different, seemingly unrelated, feature in a subsequent task. Our emphasizes that there is likely a high degree of regional specificity in the nature of neural representations, 553 with different brain networks using distinct strategies to encode task-relevant information.

555
The particular design and task set used in this experiment also reflect a targeted combination of 556 MTL related functions, including having representations related to concepts (Quiroga, 2012) and to spatial

567
One way to contextualize our findings is in relation to the notion of 'cognitive maps' (

585
This experiment emphasizes that the MTL is able to flexibly adapt to representing task-relevant 586 features across changing behavioral demands. This may also relate to a potentially surprising aspect of 587 our results, whereby we found a significant number of spatial target cells, but that we did not find a 588 significant level of place cells. This general finding, in which there is stronger evidence for spatial target 589 encoding rather than player location, is consistent with previous analyses of an independent dataset using

597
We thus hypothesize that the predominance of spatial-target responses observed here may relate to the 598 behavioral demands. Notably, the Treasure Hunt task has an emphasis on the remote location of visible 599 chest locations, rather than on current location, such that participants likely focused on the remote target 600 locations while navigating, which is reflected in the spatial target cells. This shift in the nature of location 601 tuning between place and spatial-target cells is consistent with the broader pattern we found here 602 between the one-back and spatial tasks, where human MTL neurons flexibly vary their coding properties 603 depending on task demands.

605
Beyond single-neuron recordings, our findings are also consistent with other human work that 606 has emphasized that the same circuits seem to be involved in multiple distinct feature representations 607 and tasks. Based on a review of human studies predominantly using fMRI, the human MTL has been found 608 to have a similar functional organization in relation to spatial navigation as has been established in animal 609 models, including elements of the hippocampal network that activate during specific spatial settings

622
A further probe questions about neural representations across task contexts. Although some of these 634 questions can also be examined with fMRI, which conveniently allows for multiple scans in the same 635 individuals, single-neuron recordings have the advantage of providing much higher spatial and temporal 636 precision that is useful for measuring precisely timed neuronal signals such as those that represent specific 637 locations during active navigation.

639
We also note that it is typical for human single-neuron research participants to complete multiple 640 different tasks across the 1-2 weeks that they are typically in the EMU. In principle, the kinds of

653
The medial temporal lobe is a complex structure that is known to be involved in multiple cognitive 654 processes, including spatial navigation, representing high-level concepts, and memory processing. Here,

655
we investigated the relation between these seemingly distinct functions, by using a paired task session in 656 which the activity of the same neurons can be evaluated across task contexts. By doing so, we were able 657 to show that there are multiple patterns of neural activity across tasks, whereby some neurons are active 658 only in one task context, some neurons maintain a similar representation, and some neurons appear to 659 switch their representation entirely, responding to seemingly distinct features in different task contexts.