- AssignTaskToUser
- This is where a manager acts on our reccomendation. Its not just one update; its a state change for both the task and the assigned user
- Why it’s critical transaction: If we update the task’s assignee_id but fail to update the user’s availability, our reccomendation engine will operate on stale data. It might recoomend another task to an engineer who is already busy, leading to overload. Conversely, if we mark the user as ‘busy’ but fail to link them to the task, they are effectively blocked from work for no reason
- Steps Involved
- BEGIN TRANSACTION
- UPDATE the “tasks” table:
- Set assignee_id to the selected user’s ID
- Set the status to ‘in_progress’
- UPDATE the users table
- Set the availability status to ‘busy’ for the user who was just assigned the task
- COMMIT TRANSACTION
- CompleteTask
- When a task if finished the system needs to reflect this change consistently to make the user available for new reccomendations
- This is inverse of the assignment transaction. If the task is marked as ‘done’ but the system fails to update the engineer’s status back to ‘available’, that engineer becomes a”zombie user” - permanently busy and excluded from all future reccomendations. This would cripple team productivity
- Steps Involved:
- BEGIN TRANSACTION
- Update the tasks table:
- Set the status to ‘done’
- Set the completed_at timestamp to current time
- SELECT assignee_id from the task that was just completed
- UPDATE the users table:
- Set the availability status to ‘available’ for the assignee_id retrieved in the previous step
- COMMIT TRANSACTION
- OnboardNewUserWithSkills
- A user without skills is not useful in our system. The user’s creation and the definition of their core competencies should be an atomic operation to ensure every user in the system is “reccomendable”.
- Why it’s a critical transaction: This prevents incomplete user profiles from entering the system. If we create the user record but the subsequent step of adding their skills fails, we have a user who will never be matched with any task. This creates noise in the database and could cause errors in collaborative filtering model down the line
- Steps Involved:
- BEGIN TRANSACTION
- INSERT a new record into the users table (with name, email, team_id).
- Get the id of the newly created user
- For each skill provided during onboarding (e.g., from a user signup form):
- INSERT a new record into the user_skills table, linking the new user_id with the corresponding skill_id and setting the proficiency
- COMMIT TRANSACTION
- ProcessNewTask
-
This is the entry point for our entire ML value chain. A task is created, and we immediately enrich it with the skills required to complete it
-
Why it’s a critical transaction: If we create the task in the tasks table but the NLP service fails or the skill linkage in task_required_skills fails, we’re left with an orphan task. This task has no associated skills and will be invisible to our reccomendation engine forever, defeating the core purpose of Synapse
-
Steps Involved
- BEGIN TRANSACTION
- INSERT a new record into the tasks table (with its project_id, title, description, etc.)
- In your application code, run the NLP analysis on the task’s description to extract a list of required skill names
- For each extracted skill, SELECT its id from the skills table
- For each skill_id found, INSERT a new record into the task_required_skills table, linking the new task_id with the skill_id
- COMMIT TRANSACTION
Question: What if the required skills are not present in the skills table?
Problem
In building a system that links users and tasks to relevant technical skills, we encountered a cyclic dependency in the skill extraction process:
The system needs a predefined skill dictionary to recognize and normalize extracted skills from unstructured text (e.g., task descriptions, resumes). However, this same dictionary is meant to be built and enriched over time from those unstructured inputs.
This creates a chicken-and-egg problem:
- Skill extraction depends on a dictionary,
- But building the dictionary depends on successful skill extraction.
Relying on an LLM (Large Language Model) alone for extraction and normalization is brittle—it might guess wrong or inconsistently about internal naming conventions, aliases, or database schema. Moreover, any single-step AI extraction is opaque, hard to audit, and difficult to scale or maintain.
✅ Solution Overview
To break the cycle and make the system robust, we designed a Three-Stage Skill Processing Pipeline:
Raw Extraction (LLM) → Normalization (Alias Map) → Database Write (Transactional Insert)
This approach decouples the stages of identifying, standardizing, and storing skills, allowing the system to evolve its dictionary dynamically while remaining consistent and predictable.
🧠 Stage 1: Raw Skill Extraction (LLM)
- Use the LLM only for brute-force identification of technical terms (e.g., programming languages, tools, frameworks).
- Do not ask the LLM to normalize or group aliases. This avoids guessing and ensures repeatable output.