ELLIS PHD STUDENT · UNIVERSITY OF MANCHESTER

Aditya Kapoor.

I study machine learning and how agents learn to make decisions together.

My work spans multi-agent reinforcement learning and language-model-based planning. I am expanding into probabilistic and generative modelling, including diffusion models and large language models.

I’m interested in the mathematics behind these systems, and in questions that take me beyond the methods I already know.

Download CV ↗Email me ↗Google Scholar ↗

ABOUT

About me

I’m a PhD student in Computer Science at the University of Manchester through the ELLIS programme.

I’m supervised by Omar Rivasplata, Samuel Kaski, and Jan Peters.

Previously, I was a predoctoral research fellow at TCS Research & Innovation, working on multi-agent learning, robotic perception and automatic vision-pipeline selection. I was mentored by Harshad Khadilkar, Mayank Baranwal and Jayavardhana Gubbi.

I completed my B.E. (Hons.) in Computer Science and Information Systems at BITS Pilani, Goa. My undergraduate thesis on Partial Reward Decoupling (PRD) was carried out with Benjamin Freed, Howie Choset and Jeff Schneider at Carnegie Mellon University.

During a research project at BITS Pilani, Goa (2018–2019), I worked on spiking neural networks and speech synthesis on SpiNNaker with Basabdatta Sen Bhattacharya. I also implemented the anti-Hebbian learning rule in collaboration with the Human Brain Project. It was an early chance to explore the connections between computation and neuroscience.

Alongside research, I spend time mentoring and building community projects, and exploring ways to make mathematics more inviting.

aditya.kapoor@postgrad.manchester.ac.uk GitHub ↗LinkedIn ↗

RESEARCH

Research interests

The short animations are explanations of the ideas. Use Next to walk through them at your own pace; they are not experimental recordings.

Questions I’m working on now ↓

Agents observe a shared environment. A football task is used here as an intuitive example.Each agent chooses an action: pass, move or coordinate with a teammate.The team receives feedback. Credit assignment asks which actions helped; communication asks what agents need to share.Learning updates the policies for the next attempt. The scene illustrates the questions, not a reported football result.

Multi-agent learning & decision-making

How do agents learn to work together? I study which actions deserve credit, what agents need to communicate, and how their policies can improve together.

PRD & LoRASA · DDCL & TAR²

Different data have different valid states: orientations lie on a curved space; tokens belong to a discrete set.The forward process perturbs or corrupts those states. The choice of noise depends on the space.A reverse model learns to recover structure. I’m exploring how geometry and discrete corruption shape that problem.

Probabilistic & generative modelling

How should a model learn when its data live on a curved space or take discrete values? I’m exploring diffusion, the geometry of its state space, and the role of noise in learning.

Geometry, corruption & denoising · developing work

Two strands: planning actions with language models, and studying how text is generated.LLaMAR uses execution feedback to revise plans and check progress.My newer work asks how choosing token positions affects masked-language-model generation.Published planning work and emerging generation research sit under the same broad interest in sequential decisions.

Language models: planning & generation

I work on two questions: how language models help agents plan and recover from mistakes, and when a structured decoding order should persist or change during generation.

LLaMAR · masked-model decoding

SELECTED PUBLICATIONS

Selected papers

Asterisks denote equal contribution.

The later paper brings reward relevance into multi-agent PPO.An attention-based critic estimates relevance; a soft variant weights rather than drops terms.The relevance-aware signal is used in PPO’s policy updates.A further extension handles environments with a single shared reward, unlike the original PRD formulation.

RLC / Reinforcement Learning Journal, 2024

Assigning Credit with Partial Reward Decoupling in Multi-Agent Proximal Policy Optimization

Aditya Kapoor, Benjamin Freed, Howie Choset, Jeff Schneider

This follow-up brings partial reward decoupling into multi-agent PPO. It adds soft relevance weights and a version that works when the environment provides only one shared team reward, broadening where PRD can be used.

PRD-MAPPO improved data efficiency and final performance over MAPPO on the reported benchmarks, including StarCraft II.

Paper Code

The team plans to put away the groceries: Alice handles the tomato; Bob carries the lettuce.Bob moves towards the fridge. Alice’s tomato pickup fails because she is too far away.Alice moves closer and picks up the tomato. Bob reaches the approach to the fridge with his lettuce.Alice approaches with the tomato, but Bob is in the way. She stops instead of moving into him; the dashed path and cross mark the blocked approach.Alice waits at a safe distance while Bob puts the lettuce in the fridge. They take turns using the shared approach.Bob steps aside and Alice stores the tomato. The Verifier checks both subtasks. This is a schematic example of coordination through feedback.

NeurIPS 2024, main conference

Long-Horizon Planning for Multi-Agent Robots in Partially Observable Environments

Siddharth Nayak, Adelmo Morrison Orozco, Marina Ten Have, Vittal Thirumalai, Jackson Zhang, Darren Chen, Aditya Kapoor, Eric Robinson, Karthik Gopalakrishnan, James Harrison, Brian Ichter, Anuj Mahajan, Hamsa Balakrishnan

A plan rarely survives contact with a partially observed world. LLaMAR gives a team of robots a loop for planning, acting, correcting failed actions and checking progress, using language models to work through long tasks.

Feedback and explicit checks let the team revise a plan as it discovers what actually happened.

Paper Code Project

Imagine two pairs working on separate parts of a team task.A’s learning signal can include fluctuations in rewards it could not affect.PRD estimates reward relevance and excludes unrelated terms from A’s update.That separation reduced gradient-estimator variance in the reported experiments.

IEEE Robotics and Automation Letters, 2022; presented at ICRA 2022

Learning Cooperative Multi-Agent Policies with Partial Reward Decoupling

Benjamin Freed*, Aditya Kapoor*, Ian Abraham, Jeff Schneider, Howie Choset

When a team succeeds, every agent sees outcomes it may have had little influence on. PRD learns which reward streams are relevant to each agent and leaves unrelated terms out of its learning signal. The team still works towards the same shared objective.

Removing unrelated reward terms reduced gradient noise and improved learning in the tasks we tested.

Paper Code

RobotPersonPersonal spaceInteractionObject⚑GoalRouteDirect route×Interruption

The robot must reach its goal in a space shared with people and objects.A direct route can cross an interaction even without hitting anyone.SocNavGym lets researchers compare navigation policies in configurable social scenes.

IEEE RO-MAN, 2023

SocNavGym: A Reinforcement Learning Gym for Social Navigation

Aditya Kapoor*, Sushant Swamy*, Pilar Bachiller, Luis Manso

A robot can avoid collisions and still interrupt a conversation or make people uncomfortable. SocNavGym provides configurable scenes with people, objects and social interactions so that navigation policies can be trained and compared in these situations.

The contribution is a configurable testbed for social navigation, with interactions that obstacle avoidance alone misses.

Paper Code

Train one shared policy using a multi-agent RL algorithm.Freeze the shared weights and train agent-specific low-rank matrices. Each agent can adapt without retraining the full backbone.Each policy combines shared weights with its own low-rank update. The rank controls the size of the adaptation.

DAI 2025

Low-Rank Agent-Specific Adaptation (LoRASA) for Multi-Agent Policy Learning

Beining Zhang, Aditya Kapoor, Mingfei Sun

Agents need common skills, but they do not always need identical policies. LoRASA first learns shared policy weights, then freezes them and trains small, low-rank adaptations for individual agents.

Agents can specialise around a shared policy without each needing a separate full-size network.

Paper Code

Participants learn visual categories with spatially separate features.Their descriptions indicate how many dimensions they used.More dimensions went with longer responses, supporting Combination theory within this setting.

Open Mind, 2025 · Journal article

Combination and Differentiation Theories of Categorization: A Comparison Using Participants’ Categorization Descriptions

Sujith Thomas, Aditya Kapoor, Narayanan Srinivasan

Does categorisation begin with a whole impression, or by combining individual features? We used participants’ descriptions of their choices alongside response times to compare these accounts.

Using more dimensions took longer, supporting Combination theory for the spatially separable visual features studied.

Paper

PREPRINTS

Preprints

Public manuscripts; listed separately from peer-reviewed publications.

In the paper’s goal-finding example, the speaker sees the target and must inform a listener that can move.The message signal is perturbed with shared dither and quantised into a discrete message. The channel handles signed, unbounded signals.The receiver reconstructs a signal and subtracts the shared dither. The resulting channel admits a noise-based training formulation.Training balances the task objective with a differentiable communication penalty, encouraging economical messages. This is a mechanism sketch, not a measured bit allocation.

arXiv preprint, 2025; under review

Learning what to say and how precisely: Efficient Communication via Differentiable Discrete Communication Learning

Aditya Kapoor, Yash Bhisikar, Benjamin Freed, Jan Peters, Mingfei Sun

What should an agent communicate when every bit has a cost? We introduce Differentiable Discrete Communication Learning (DDCL), a framework for learning discrete messages. It supports signed, unbounded message vectors, using shared dithering and a differentiable communication penalty to balance message cost and task performance.

The message values are learned; the quantisation granularity is fixed. The method trades communication cost against task performance.

Paper Code

A pass, a run and a shot contribute at different moments. This is an illustrative episode.The environment supplies a delayed team reward. Earlier actions have no immediate feedback.TAR² models the episode to allocate reward across both agents and timesteps.The allocated rewards train the policies. Shares shown here are conceptual, not measured or causal attributions.

arXiv preprint, 2025; under review

Redistributing Rewards Across Time and Agents for Multi-Agent Reinforcement Learning

Aditya Kapoor, Kale-ab Tessera, Mayank Baranwal, Harshad Khadilkar, Jan Peters, Stefano Albrecht, Mingfei Sun

A reward at the end of an episode says little about which agents helped, or when. TAR² uses the recorded team trajectory to distribute that reward across agents and earlier timesteps, giving policy learning a more detailed signal.

Jointly allocating credit across time and agents improved learning in the sparse-reward benchmarks studied.

Paper

IN PROGRESS

Questions I’m working on now

These questions connect my work on learning in teams with my growing interest in probabilistic and generative models.

Research in progress

Learning communication precision

How much detail does a teammate need to make a good decision? I’m studying when learning message precision improves on fixed precision or a prescribed schedule, at a given communication cost. The work also asks how message scale, reconstruction and optimisation shape that trade-off.

Related paper: DDCL, which uses fixed grid spacing.

An agent sends information that a teammate uses to make a decision. More detail can cost more bits.The follow-on work asks whether agents can learn a useful precision, shared across a message or chosen per coordinate.Learning must be compared with a well-chosen fixed precision and with a prescribed schedule, using measured communication cost.The question is when learning precision improves the trade-off. Message scale, reconstruction and optimisation can all affect the answer.

Research in progress

Credit assignment for policy learning

Splitting a team’s reward is only useful if the resulting learning signals help its agents improve. I’m studying when redistribution preserves the direction of each agent’s policy gradient, and how hindsight information can provide credit aligned with the team objective.

Related work: TAR² and PRD-MAPPO.

A team receives a return after several agents have acted. Reward redistribution gives individual actions a learning signal.Keeping the same total return does not by itself ensure that each agent learns in a direction that helps the team.I’m studying when hindsight information can produce local learning signals whose expected policy gradients align with the team objective.

Research in progress

Simultaneous multi-agent policy updates

When several agents learn at once, two useful changes can work against one another. I’m studying whether measured interactions between proposed updates can guide how far each agent moves, and when this can guarantee improvement in the team’s performance.

When should a decoding order change?

A masked language model can fill in text in different orders. Should it keep one structured order across the text, or redraw it between parts? I’m studying when each choice helps, comparing likelihood and generated-text quality separately at the same inference cost.

A structured decoding order chooses which positions to generate together. Different orders reveal different context.One option is to keep the same order across successive parts of a text.Another is to redraw the order between parts. A redraw can select the same order again.I’m studying when persistence or redrawing helps, checking likelihood and generated-text quality separately at matched inference cost.

Research in progress

Geometry, noise and generative models

I’m exploring how to steer diffusion models towards useful outcomes while respecting the geometry of their state space. Alongside this, I’m studying dithering in the forward corruption of discrete generative models: how does the choice of corruption shape what a model can learn to reverse?

For structured data, such as orientations, the space of valid states matters.Noise and denoising need to respect the kind of data being modelled.This sketch shows noising and reconstruction. My current questions include reward-guided sampling on curved spaces and dithering in discrete corruption.

I’m also exploring LLM quantisation and watermarking, including possible uses of dithering. These are early interests. If any of these questions overlap with yours, I’d be happy to talk.

INTERESTS

What I’m learning

There is a lot I would like to learn beyond my current research. Information theory, geometry and topology are some mathematical interests; neuroscience, cognitive science and psychology are others. The list keeps growing.

I hope to learn some of these subjects through research and some simply by reading and talking to people who know more about them.

At the moment, I’m spending more time on probabilistic and generative modelling, including masked language models and diffusion on curved spaces.

If you’re working on a related question, I’d be happy to talk — send me an email.

More publications, preprints and patent applications

MAP-THOR defines household tasks with different amounts of information stated explicitly in the instruction.Multiple floor plans test whether a planner can handle changes in layout under partial observability.Planning systems can be compared using success, transport and coverage, among other metrics. No measured performance is shown here.

ICML 2024, Multi-modal Foundation Model meets Embodied AI Workshop

MAP-THOR: Benchmarking Long-Horizon Multi-Agent Planning Frameworks in Partially Observable Environments

Long household tasks are a useful test of whether several robots can coordinate with incomplete information. MAP-THOR provides tasks and evaluation for comparing planning frameworks in this setting; this workshop work is part of the LLaMAR research line.

A shared benchmark makes it possible to compare how planners handle coordination and partial observability.

Paper Code

A late team reward leaves both the responsible agents and useful time steps unclear.The workshop method models temporal and inter-agent structure to learn reward-redistribution weights.Weights allocate the episodic signal across agent–time pairs. This diagram does not show measured allocations.The resulting signal supports policy learning. The visual makes no claim of guaranteed policy preservation.

RLC 2024, Coordination and Cooperation for Multi-Agent Reinforcement Learning Methods Workshop

Agent-Temporal Credit Assignment for Optimal Policy Preservation in Sparse Multi-Agent Reinforcement Learning

Aditya Kapoor, Sushant Swamy, Mayank Baranwal, Mingfei Sun, Harshad Khadilkar, Stefano V Albrecht

This workshop paper asks how a delayed team reward can become useful feedback for individual agents at earlier moments. It is the precursor to the reward-redistribution work developed further in TAR².

The central idea is to consider the agent and the timestep together when assigning credit.

Paper

Detect shelf products and compare cropped objects within a row.A concept-learning representation separates colour and shape/content; outlier detection identifies an item that does not fit its context.The anomaly feeds a task planner and robot motion planning for correction. The drawings are a simplified example.

IEEE Systems, Man, and Cybernetics Society (SMCS), 2023

Concept-based Anomaly Detection in Retail Stores for Automatic Correction using Mobile Robots

Aditya Kapoor, Vartika Sengar, Nijil George, Vighnesh Vatsal, Jayvardhana Gubbi, Balamuralidhar P, Arpan Pal

A misplaced item is easier to correct if a robot can describe what is wrong. This work detects anomalies through concepts such as colour and shape, then connects that description to a plan for correcting the arrangement.

Concept-based descriptions connect visual anomaly detection to actions a robot can carry out.

Paper

Retail robots must interpret cluttered scenes and the objects in them.They also need to move through spaces shared with people.Physical manipulation adds another challenge. The paper discusses these connected real-world difficulties.

ICRA 2022 Workshop - Challenges in Applying Academic Research to Real-World Robotics

Challenges in Applying Robotics to Retail Store Management

Vartika Sengar*, Aditya Kapoor*, Nijil George*, Vighnesh Vatsal*, Jayvardhana Gubbi, Balamuralidhar P, Arpan Pal (* indicates equal contribution)

Taking a robot from a demonstration to a shop exposes gaps between perception, navigation and manipulation. This paper reflects on the practical challenges of putting these pieces together for retail-store tasks.

Reliable deployment depends on how the components work together, as well as how each performs on its own.

Paper

The high-level vision stages are supplied in advance; the problem is to choose the algorithm within each stage.State-attribute identifiers narrow the choices, while transformer-based policies select algorithms.Classification performance provides feedback for PPO training. The selected options here are illustrative.

ICRA 2022 Workshop - Robotic Perception and Mapping: Emerging Techniques

Auto-TransRL: Autonomous Composition of Vision Pipelines for Robotic Perception

Aditya Kapoor, Vartika Sengar, Nijil George, Vighnesh Vatsal, Jayvardhana Gubbi

A vision pipeline often offers several algorithms for each processing stage. Auto-TransRL uses reinforcement learning to choose among them within a given stage structure, adapting the pipeline to the perception task.

Algorithm selection becomes a learned decision instead of a single fixed choice for every input.

Paper

Train categories with perfectly and partially diagnostic features.Measure participants’ knowledge of the feature–category associations.Partial cues were used when learned very accurately, helping explain differences in categorisation strategy.

CogSci 2021

Supervised category learning: When do participants use a partially diagnostic feature?

Sujith Thomas, Aditya Kapoor, Narayanan Srinivasan

People often classify an object using the one feature that reliably identifies its category. We asked when they also use features that are informative but imperfect, measuring how well those features had been learned.

Participants used partial cues when they knew them very accurately; cue knowledge helped explain the strategy they chose.

Paper

SONATA creates configurable simulated scenes containing people, objects, a robot and social relationships.A controller combines simulation feedback with human input, such as teleoperation or labels.Scene geometry, relationships and human-provided information are recorded to build a task-specific dataset.

Workshop of Physical Agents 2020

A toolkit to generate social navigation datasets

Rishabh Baghel, Aditya Kapoor, Pilar Bachiller, Ronit R Jorvekar, Daniel Rodriguez-Criado, Luis J Manso

Learning social navigation needs examples of how a robot moves around people. This toolkit generates simulated scenes and records demonstrations and scene relationships for building navigation datasets.

Reusable data-generation tools make social-navigation experiments easier to set up and reproduce.

Paper

Keep the underlying category structure unchanged.Vary serial versus concurrent presentation, and the presence of a colour-based descriptor.Serial presentation elicited fewer dimensions; presentation and colour descriptors both affected categorisation.

CogSci 2020

Effect of a colour-based descriptor and stimuli presentation mode in unsupervised categorization

Sujith Thomas, Aditya Kapoor, Narayanan Srinivasan

We kept the logical category structure fixed while changing how items were presented and whether colour supplied an additional descriptor. This lets us ask how presentation changes a seemingly natural way of grouping things.

Serial presentation led to fewer dimensions being used. Both presentation mode and colour descriptors influenced categorisation.

Paper

An image can contain more than one distortion, such as noise and an exposure change.DeepClean first identifies the correction type rather than assuming a fixed sequence of image-processing steps.It selects an algorithm for the chosen correction type using candidate output features.After applying the correction, the updated image is assessed again; a no-correction decision stops the sequence. This is a mechanism sketch.

arXiv preprint

DeepClean: Integrated Distortion Identification and Algorithm Selection for Rectifying Image Corruptions

Aditya Kapoor, Harshad Khadilkar, Jayvardhana Gubbi

An image may suffer from more than one kind of corruption. DeepClean combines distortion identification with algorithm selection, choosing a correction and then reassessing the image before deciding what to do next.

Choosing and ordering corrections together makes the cleaning process responsive to the image.

Paper

The described system extracts object concepts and spatial relationships from visual input.Interpretable symbolic representations connect the detected anomaly with a corrective goal.Task and motion planning turn the goal into robot actions. This is an overview of the patent application, not a recorded demonstration.

US patent · granted May 2025

Systems and methods for anomaly detection and correction

Jayavardhana Rama Gubbi Lakshminarasimha, Vartika Sengar, Vighnesh Vatsal, Balamuralidhar Purushothaman, Arpan Pal, Nijil George, Aditya Kapoor

This patent covers systems for detecting and correcting anomalies, arising from our collaborative work on robotic perception and correction.

It connects identifying an anomaly with choosing a corrective action.

US patent · granted May 2025

A descriptive query and image state attributes specify the vision task.A symbolic planner identifies stages; a transformer and RL framework selects suitable algorithms.The selected algorithms form an executable pipeline. This is a conceptual overview of the patent application.

Patent application

Methods and systems for autonomous task composition of vision pipelines using an algorithm selection framework

Abhishek Roy Choudhury, Vighnesh Vatsal, Mehesh Rangarajan, Naveen Kumar Basa Anitha, Aditya Kapoor, Jayavardhana Rama Gubbi Lakshminarasimha, Aravindhan Saravanan, Vartika Sengar, Balamuralidhar Purushothaman, Arpan Pal, Nijil George

This application describes an algorithm-selection framework for composing vision pipelines for a task, arising from our collaborative perception research.

The organising idea is to choose processing algorithms according to the task and pipeline context.

Patent application

MENTORING & BUILDING

Mentoring & student projects

Open source & research mentoring

I was a Google Summer of Code contributor with RoboComp in 2021, working on Improvement of a Social Navigation component with mentors Luis J. Manso and Pilar Bachiller. I returned as a mentor in 2022, supporting Sushant Swamy’s project, Extending the Social Navigation component to realistic scenarios. That mentoring developed into SocNavGym. I was also a founding member of CORD.ai, a student organisation where I helped build the research programme and led student recruitment and ongoing mentoring. We worked together on research questions, methods and papers.

I’ve since written papers with former mentees, including Sushant Swamy and Yash Bhisikar. Seeing those relationships turn into collaborations has been one of the pleasures of mentoring.

Project Kratos

As an undergraduate at BITS Pilani, Goa, I was part of the founding team of Project Kratos, our student Mars rover project. I worked on the autonomous navigation stack and path planning, including an RRT* variant, with Ojit Mehta from BITS Goa. I later helped mentor the navigation team. We raised the funds ourselves and built our first rover within a budget of ₹1 lakh (₹100,000), finishing 10th out of 35 teams on our first attempt.

COMMUNITY & EDUCATION

Community & education

StraysCue

I run StraysCue, a nonprofit initiative that helps rescue stray animals in urban India, including Mumbai. I lead rescue coordination, veterinary-care arrangements, partnerships and fundraising, working with others on resources and day-to-day support.

Agla Station

We recently started Agla Station to help students in Classes 9 and 10 in rural India discover educational and career possibilities. The aim is to make a wider range of paths visible while students are beginning to think about their next steps. I help develop the programme and its relationships with schools, and lead follow-up support.

Mathematics through play

I’m exploring how origami, magic, Rubik’s cubes, dance and other creative activities could help children engage with mathematics.

I’m currently developing the idea and reading about mathematics education and learning, with the aim of making mathematical exploration more playful and approachable.

BEYOND RESEARCH

Outside research

01 / ON THE PITCH

Football & futsal

I played in Goa League Division 2 with BITS Goa FC, the only college team in the Goa Football Association league. We qualified for Division 1 in 2020. I currently play futsal for the University of Manchester.

I also enjoy a game of badminton.

A folded paper crane, puzzle cube and chess rook

02 / MAKING & THINKING

Paper, puzzles & play

Away from sport, you might find me folding origami or playing chess. I’m also exploring origami, magic, Rubik’s cubes and dance as ways to teach mathematics.

A mountain trail, gymnastic rings and a surfskate

03 / OUT & ABOUT

Movement & mountains

I enjoy surfskating, calisthenics, gymnastics, bouldering and hiking.

I have unfinished business with the Welsh 3K hiking challenge. I’d like to go back and finish it.

An open book with a bookmark and glasses

04 / BETWEEN THE PAGES

Reading beyond my field

I enjoy reading books and have been getting into philosophy and politics to understand the world a little better.

There is a lot I’d like to learn outside the subjects I work on.