Users Backend

1. Overview

The API Users service is a microservice responsible for managing user accounts, authentication, authorization, and user-related operations within the Nimbly ecosystem. Built with Node.js and Express, it leverages Firebase for authentication and MongoDB for data persistence.

Key Features

User authentication (login, refresh token, password reset)
User management (CRUD operations)
Role-based [access control](../Settings/Access control/AccessControlOverview.md)
Organization-based multi-tenancy
Department and site assignment
Push notification management
User activity tracking and metrics

2. Architecture Overview

graph TB
    subgraph "Client Layer"
        Mobile[Mobile Apps]
        Web[Web Apps]
        Internal[Internal Services]
    end

    subgraph "API Gateway"
        Router[User Router]
    end

    subgraph "Business Logic"
        Controller[User Controller]
        UseCase[User UseCase]
    end

    subgraph "Data Access"
        FirebaseRepo[Firebase Repository]
        MongoRepo[MongoDB Repository]
        DeptRepo[Department Repository]
        ActivityRepo[Activity Repository]
    end

    subgraph "External Services"
        Firebase[Firebase Auth]
        MongoDB[(MongoDB)]
        CloudTasks[Cloud Tasks]
        SendGrid[SendGrid Email]
        PubSub[Pub/Sub]
    end

    Mobile --> Router
    Web --> Router
    Internal --> Router

    Router --> Controller
    Controller --> UseCase

    UseCase --> FirebaseRepo
    UseCase --> MongoRepo
    UseCase --> DeptRepo
    UseCase --> ActivityRepo

    FirebaseRepo --> Firebase
    MongoRepo --> MongoDB
    UseCase --> CloudTasks
    UseCase --> SendGrid
    UseCase --> PubSub

3. Libraries and Modules

Core Framework

express (^4.17.1) - Web application framework
@google-cloud/functions-framework (~1.9.0) - Google Cloud Functions support
cors (^2.8.5) - Cross-origin resource sharing

Authentication & Security

firebase (^8.3.2) - Firebase SDK for authentication
firebase-admin (9.4.2) - Firebase Admin SDK
jsonwebtoken (^8.5.1) - JWT token generation and validation
bcryptjs (^2.4.3) - Password hashing

Data Layer

mongoose (5.12.3) - MongoDB ODM
mongoose-paginate-v2 (^1.3.17) - Pagination plugin
redis (^3.0.2) - Caching layer

Validation & Utilities

joi (^17.3.0) - Request validation
lodash (^4.17.20) - Utility functions
moment (^2.29.1) - Date manipulation
moment-timezone (^0.5.32) - Timezone handling

External Services

@google-cloud/tasks (^2.3.0) - Cloud Tasks integration
@google-cloud/trace-agent (^5.1.3) - Application tracing
axios (^0.21.1) - HTTP client

Internal Packages

@nimbly-technologies/nimbly-common - Shared models and types
@nimbly-technologies/nimbly-backend-utils - Backend utilities
@nimbly-technologies/nimbly-access - Access control
@nimbly-technologies/entity-node - Entity management

4. API Endpoints

4.1. Authentication Endpoints

Method	Endpoint	Description	Auth Required
POST	`/login`	User login with email/password	No
POST	`/v2/login`	Enhanced login with level info	No
POST	`/refresh-token`	Refresh access token	No
POST	`/activate-user`	Activate fresh user account	No
POST	`/reset-password`	Send password reset email	No
POST	`/confirm-password`	Confirm password reset	No
POST	`/logout`	Logout user	Yes

4.2. User Management Endpoints

Method	Endpoint	Description	Auth Required
GET	`/`	Get all users in organization	Yes
GET	`/:userID`	Get specific user by ID	Yes
GET	`/:userID/populated`	Get user with dept/site data	Yes
POST	`/`	Create new user	Yes
POST	`/update`	Update user information	Yes
PUT	`/toggle-status/:userID`	Enable/disable user	Yes
POST	`/profile-photo`	Update profile photo	No
PUT	`/download-modes`	Update download preferences	No

4.3. Organization & Role Endpoints

Method	Endpoint	Description	Auth Required
GET	`/organizations/:organizationID`	Get users by organization	Yes
GET	`/supervisor/:supervisorID`	Get users by supervisor	Yes
POST	`/baseRole`	Find users by base roles	Yes
POST	`/roles`	Find users by specific roles	Yes
GET	`/getRoles`	Get roles in organization	Yes
POST	`/update-users-role`	Bulk update user roles	Yes

4.4. Query & Filter Endpoints

Method	Endpoint	Description	Auth Required
GET	`/paginate`	Get paginated users	Yes
GET	`/compact`	Get compact user list	Yes
POST	`/user-dept`	Get users by department	Yes
POST	`/check-availability`	Check email availability	Yes
GET	`/overview`	User statistics overview	Yes

4.5. Utility Endpoints

Method	Endpoint	Description	Auth Required
GET	`/getUsers`	CS utility - get users	Yes
POST	`/delete-multiple-users`	CS utility - bulk delete	Yes
POST	`/sendNotification`	Send push notifications	Yes
POST	`/download-auditors`	Download auditor data	Yes

4.6. Metrics Endpoints

Method	Endpoint	Description	Auth Required
POST	`/user-activity`	Get user activity metrics	Yes
POST	`/login-platform`	Get login platform stats	Yes
POST	`/user-stats`	Get comprehensive user stats	Yes

5. Authentication and Authorization

5.1. Authentication Flow

sequenceDiagram
    participant Client
    participant Router
    participant Controller
    participant UseCase
    participant Firebase
    participant MongoDB

    Client->>Router: POST /login {email, password}
    Router->>Controller: login()
    Controller->>UseCase: performLogin()
    
    UseCase->>Firebase: signInWithEmailAndPassword()
    Firebase-->>UseCase: firebaseToken
    
    UseCase->>MongoDB: findByEmail()
    MongoDB-->>UseCase: userDocument
    
    UseCase->>UseCase: createLoginResponse()
    UseCase-->>Controller: {user, token, expiresIn}
    Controller-->>Router: response
    Router-->>Client: 200 OK {data}

5.2. Middleware Chain

graph LR
    Request[HTTP Request] --> AuthMiddleware[Auth Middleware]
    AuthMiddleware --> ValidatorHandler[Validator Handler]
    ValidatorHandler --> ExpressMiddleware[Express Middleware]
    ExpressMiddleware --> Controller[Controller Method]

5.3. Authorization Levels

No Auth Required
- Login endpoints
- Password reset endpoints
- Profile photo updates
Basic Auth Required
- All user query endpoints
- User creation/update
- Notification sending
Role-Based Auth
- SUPERADMIN: Overview stats, internal operations
- ACCOUNT_HOLDER: Organization-wide operations
- Regular users: Department-scoped operations

6. Data Models

6.1. User Schema

classDiagram
    class User {
        +String userID
        +String email
        +String displayName
        +String phoneNumber
        +String photoURL
        +String username
        +String password
        +String role
        +String organizationID
        +String status
        +String prevStatus
        +String authType
        +String language
        +String downloadModes
        +String whatsappNumber
        +Boolean whatsappAdvanceNotification
        +String uuid
        +DeviceInfo deviceInfo
        +Date createdAt
        +String createdBy
        +Date updatedAt
        +String updatedBy
    }

    class DeviceInfo {
        +String binaryVersion
        +String carrier
        +String locale
        +String phoneBrand
        +String phoneModel
        +String phoneOS
        +String phoneOSVersion
    }

    User --> DeviceInfo

6.2. User Status Lifecycle

stateDiagram-v2
    [*] --> Fresh: User Created
    Fresh --> Active: Activate User
    Fresh --> Disabled: Admin Disable
    Active --> Disabled: Toggle Status
    Disabled --> Active: Toggle Status
    Active --> [*]: Delete User
    Disabled --> [*]: Delete User

7. Core Flows

7.1. User Creation Flow

sequenceDiagram
    participant Admin
    participant Controller
    participant UseCase
    participant Validator
    participant Firebase
    participant MongoDB
    participant CloudTasks

    Admin->>Controller: POST /users
    Controller->>Validator: Validate input
    Validator-->>Controller: Valid

    Controller->>UseCase: create(context, orgID, userData)
    
    UseCase->>UseCase: checkExistingUser()
    UseCase->>UseCase: validatePassword()
    UseCase->>UseCase: generateAuthType()
    
    UseCase->>Firebase: createUser()
    Firebase-->>UseCase: firebaseUser
    
    UseCase->>MongoDB: saveUser()
    MongoDB-->>UseCase: savedUser
    
    UseCase->>UseCase: assignDepartments()
    UseCase->>UseCase: assignSites()
    
    UseCase->>CloudTasks: scheduleWelcomeEmail()
    
    UseCase-->>Controller: {user, message}
    Controller-->>Admin: 201 Created

7.2. Password Reset Flow

sequenceDiagram
    participant User
    participant Controller
    participant UseCase
    participant Firebase
    participant CloudTasks
    participant Email

    User->>Controller: POST /reset-password
    Controller->>UseCase: sendResetPassword(email)
    
    UseCase->>UseCase: checkUser(email)
    UseCase->>UseCase: validateUserStatus()
    
    UseCase->>CloudTasks: newUserEmailTask()
    CloudTasks->>Email: Send reset email
    
    UseCase-->>Controller: Success message
    Controller-->>User: 200 OK

    Note over User: User receives email

    User->>Controller: POST /confirm-password
    Controller->>UseCase: confirmResetPassword(password, oobCode)
    
    UseCase->>Firebase: confirmPasswordReset()
    Firebase-->>UseCase: Confirmation
    
    UseCase->>UseCase: updateUserStatus(ACTIVE)
    
    UseCase-->>Controller: Success
    Controller-->>User: 200 OK

7.3. Department Assignment Flow

flowchart TB
    Start([User Update Request]) --> ValidateDepts{Validate Departments}
    ValidateDepts -->|Invalid| Error[Return Error]
    ValidateDepts -->|Valid| GetCurrent[Get Current Assignments]
    
    GetCurrent --> Compare[Compare Old vs New]
    Compare --> RemoveOld[Remove Old Assignments]
    Compare --> AddNew[Add New Assignments]
    
    RemoveOld --> UpdateIndex1[Update Dept Index]
    AddNew --> UpdateIndex2[Update Dept Index]
    
    UpdateIndex1 --> Activity1[Log Activity]
    UpdateIndex2 --> Activity2[Log Activity]
    
    Activity1 --> Complete[Update Complete]
    Activity2 --> Complete

8. Implementation Details

The login process is a sophisticated multi-step authentication flow designed to handle various authentication scenarios while maintaining security and performance. The implementation follows a clear separation between authentication logic and response preparation.

Authentication Flow Architecture

The login system is divided into three main phases:

Phase 1: Initial Authentication (doLogin) The system first determines the authentication method based on the user’s email domain. For organizations with custom domains, it routes through federated authentication providers (SAML, OAuth), while standard domains use Firebase authentication. This determination happens by checking the email against a pre-configured list of federated domains stored in the organization’s settings.

During Firebase authentication, the system performs email/password validation against Firebase Auth service. If successful, it receives a Firebase ID token containing the user’s authentication claims. For federated authentication, the system validates the external identity provider’s token and maps it to internal user credentials.

Phase 2: User Validation and Enrichment After successful authentication, the system retrieves the complete user profile from MongoDB. This includes not just basic user data but also their role assignments, department memberships, and site associations. The system performs critical status checks to ensure the user account is active. Disabled or suspended accounts are rejected at this stage with appropriate error messages.

The validation process also checks for account-specific restrictions such as:

Expiration dates for temporary accounts
Geographic restrictions based on login location
Device restrictions for high-security accounts
Time-based access controls for shift workers

Phase 3: Response Construction (createLoginResponse) The final phase constructs a comprehensive authentication response. This involves:

Token Generation: Creates a JWT token with embedded user claims, organization context, and permission scopes. The token includes a unique session identifier for tracking and revocation purposes.
Refresh Token Creation: Generates a long-lived refresh token stored in a separate collection with device fingerprinting. This allows for secure token renewal without re-authentication.
Permission Loading: Retrieves the user’s effective permissions by merging role-based permissions, department-specific access rights, and any user-specific overrides. This creates a complete permission matrix used throughout the session.
Department Context: Loads all departments the user has access to, including their role within each department. This hierarchical access model allows users to have different permissions in different organizational units.
Extension Data: Fetches user-specific extensions such as UI preferences, notification settings, and custom attributes defined by the organization.

8.2. User Creation Implementation

The user creation process is a complex orchestration of multiple systems that ensures data consistency across Firebase Authentication and MongoDB while maintaining proper access controls and audit trails. The implementation employs a careful sequence of operations with rollback capabilities to handle failures gracefully.

Creation Process Architecture

Step 1: Email Uniqueness Validation The system performs a multi-layer uniqueness check across both Firebase and MongoDB. This dual-check approach prevents race conditions where a user might exist in one system but not the other. The validation includes:

Case-insensitive email matching to prevent duplicate accounts with different casing
Checking against both active and soft-deleted users to prevent resurrection of deleted accounts
Validation against organization-specific email rules and blacklists

Step 2: Authentication Type Determination Based on the email domain and organization configuration, the system determines the appropriate authentication mechanism. Organizations can configure:

Standard Firebase authentication for general users
Federated authentication for corporate domains
Mixed-mode authentication where certain email patterns use different providers
Custom authentication flows for specialized security requirements

Step 3: Firebase Account Creation The Firebase account creation involves more than just setting email and password. The system:

Generates a secure password following organization-specific complexity rules
Sets custom claims including organization ID, initial role, and metadata
Configures email verification requirements based on organization policy
Establishes multi-factor authentication settings if required

Step 4: MongoDB User Document Creation The user document in MongoDB serves as the primary source of truth for user data. The creation process:

Generates a unique user ID that remains constant across systems
Sets initial status to “fresh” indicating the account needs activation
Populates default values for language, timezone, and notification preferences
Creates audit fields tracking who created the account and when
Establishes links to the Firebase UID for authentication correlation

Step 5: Department and Site Assignment This critical step establishes the user’s organizational access. The system:

Validates that all requested departments exist and are active
Checks that the creating user has permission to assign users to these departments
Creates entries in the department index for efficient querying
Establishes role hierarchies within each department
Sets up site-specific permissions including supervisor and auditor roles
Maintains bi-directional references for data integrity

Step 6: Welcome Email and Onboarding The final step initiates the user onboarding process:

Schedules a welcome email through the Cloud Tasks queue for reliable delivery
Includes activation links with time-limited tokens
Personalizes content based on assigned role and departments
Triggers any organization-specific onboarding workflows
Creates initial notification subscriptions based on role

8.3. Pagination Implementation

The pagination system is designed to handle large datasets efficiently while providing flexible querying capabilities. The implementation goes beyond simple limit/offset pagination to provide advanced features for complex user interfaces.

Advanced Pagination Features

Dynamic Query Building The pagination system constructs MongoDB queries dynamically based on multiple filter parameters. It intelligently combines:

Text search across multiple fields with relevance scoring
Date range filters for temporal queries
Status filters with support for multiple selections
Role-based filtering with hierarchical inheritance
Department-scoped queries respecting access controls

Performance Optimization The system employs several optimization strategies:

Index hints to force optimal query plans
Projection limiting to reduce data transfer
Cursor-based pagination for consistent results during data changes
Aggregation pipeline optimization for complex sorting requirements
Result caching for frequently accessed pages

Metadata Generation Each paginated response includes comprehensive metadata:

Total count with smart caching to avoid repeated counts
Page information including current, total, and navigation links
Sort indicators showing active sort fields and directions
Filter summary showing applied filters for UI state management
Performance metrics including query execution time

8.4. Activity Tracking

The activity tracking system provides comprehensive audit trails for compliance and debugging purposes. It captures not just what changed, but the complete context of how and why changes occurred.

Activity Tracking Architecture

Event Capture The system automatically captures activities at multiple levels:

API level: All HTTP requests with parameters and responses
Business logic level: Significant operations and their outcomes
Data level: All create, update, and delete operations with snapshots
Integration level: External service calls and their results

Context Enrichment Each activity record is enriched with contextual information:

User context including role, department, and permissions at time of action
Session information including IP address, user agent, and geographic location
Request context including API version, client application, and correlation IDs
Temporal context including timestamps in multiple timezones

Snapshot Management For data modifications, the system creates detailed snapshots:

Before state: Complete object state before modification
After state: Complete object state after modification
Diff generation: Computed differences highlighting exactly what changed
Relationship tracking: Changes to related objects are linked

Activity Storage Activities are stored using a multi-tier approach:

Hot storage: Recent activities in MongoDB for quick access
Warm storage: Older activities in compressed format
Cold storage: Archived activities in cloud storage for compliance
Real-time stream: Activities published to Pub/Sub for real-time monitoring

8.4. Department Access Control

flowchart LR
    User[User] --> Check{Has Dept Access?}
    Check -->|Yes| LoadDepts[Load Departments]
    Check -->|No| EmptyList[Return Empty]
    
    LoadDepts --> Filter[Filter by Status]
    Filter --> Map[Map Permissions]
    Map --> Return[Return Access List]

8.5. Error Handling Patterns

The error handling system implements a sophisticated multi-layer approach that provides meaningful feedback while maintaining security. The architecture ensures that errors are properly caught, logged, and transformed at each layer of the application.

Error Handling Architecture

Controller Layer Error Handling At the controller level, errors are caught and transformed into standardized HTTP responses. The system distinguishes between:

Expected business errors that should be communicated to users
Unexpected system errors that should be logged but hidden from users
Validation errors that need detailed field-level feedback
Authentication errors that require special handling to prevent information leakage

Use Case Layer Error Management The use case layer implements domain-specific error handling that:

Transforms technical errors into business-meaningful messages
Aggregates multiple errors from different operations
Implements retry logic for transient failures
Manages transaction rollbacks on critical errors
Provides detailed error context for debugging

Repository Layer Error Handling At the data access layer, the system:

Catches database-specific errors and normalizes them
Implements timeout handling for long-running queries
Manages connection failures with circuit breaker patterns
Logs query details for performance debugging
Handles concurrent modification conflicts

8.6. Validation Middleware

The validation system provides comprehensive request validation using a declarative schema approach. This multi-stage validation ensures data integrity at the API boundary.

Validation Architecture

Schema-Based Validation The Joi validation schemas define:

Field-level constraints including type, length, and format
Cross-field validations for business rules
Conditional validations based on user roles or request context
Custom validation functions for complex business logic
Error message customization for user-friendly feedback

Validation Pipeline The validation process follows a strict pipeline:

Syntax Validation: Ensures request structure is valid JSON/form-data
Schema Validation: Applies Joi schemas to validate data types and constraints
Business Validation: Executes custom validators for business rules
Security Validation: Checks for injection attempts and malicious patterns
Context Validation: Ensures request is valid for user’s current context

Error Response Formatting Validation errors are formatted to provide:

Field-specific error messages for form rendering
Error codes for programmatic handling
Suggestions for fixing common validation issues
Localized error messages based on user preferences
Detailed debugging information in development mode

9. Repository Implementation Patterns

9.1. MongoDB Repository Architecture

The MongoDB repository implements advanced patterns for data access:

classDiagram
    class IUserRepository {
        <<interface>>
        +findByEmail(email)
        +findByID(userID)
        +findAll(query)
        +create(userData)
        +update(userID, data)
        +delete(userID)
        +paginate(query, options)
    }

    class UserMongoRepository {
        -model: Model
        -connection: Connection
        +findByEmail(email)
        +findByID(userID)
        +findAllWithAggregation(pipeline)
        +bulkOperations(operations)
        +transaction(callback)
    }

    class BaseRepository {
        <<abstract>>
        #handleError(error)
        #logQuery(query)
        #withTimeout(promise, timeout)
    }

    IUserRepository <|.. UserMongoRepository
    BaseRepository <|-- UserMongoRepository

Key Repository Features

Cursor-Based Iteration
- Efficient memory usage for large datasets
- Streaming data processing
- Automatic cleanup on errors
Aggregation Pipelines
- Complex joins with $lookup
- Data transformation with $project
- Performance optimization with $match early
Transaction Support
- Multi-document ACID transactions
- Rollback on failure
- Session management

9.2. Firebase Repository Architecture

The Firebase repository provides real-time capabilities:

sequenceDiagram
    participant UseCase
    participant FirebaseRepo
    participant FirebaseAuth
    participant FirebaseDB
    participant Cache

    UseCase->>FirebaseRepo: createUser()
    FirebaseRepo->>FirebaseAuth: createUserWithEmailAndPassword()
    FirebaseAuth-->>FirebaseRepo: authUser
    
    FirebaseRepo->>FirebaseDB: set(/users/{uid})
    FirebaseDB-->>FirebaseRepo: success
    
    FirebaseRepo->>Cache: invalidate(user:{uid})
    Cache-->>FirebaseRepo: cleared
    
    FirebaseRepo-->>UseCase: User

10. Conclusion

The API Users service provides a robust, scalable, and secure user management system with comprehensive authentication, authorization, and user lifecycle management capabilities. Its modular architecture, extensive validation, and integration with cloud services make it a critical component of the Nimbly platform.

Key strengths include:

Clean architecture with clear separation of concerns
Comprehensive security measures with RBAC
Flexible multi-tenant authentication
Scalable cloud-native design
Extensive monitoring and observability
Advanced caching and performance optimization
Comprehensive testing strategies

Areas for potential enhancement:

GraphQL API support for flexible querying
Enhanced caching with Redis clustering
Real-time user presence with WebSockets
Advanced analytics with BigQuery integration
Multi-factor authentication (MFA)
Passwordless authentication options
User impersonation for support
Advanced audit trail with blockchain

Compass - Nimbly Documentation

Explorer