Compass - Nimbly Documentation

Bulk Operation Frontend V2

20 min read

1. Overview

The Bulk Operations module is a comprehensive system designed to handle large-scale data manipulation within the audit administration platform. It provides two primary functionalities:

  1. Bulk Edit: Allows administrators to download existing data in Excel format, make modifications, and re-upload for batch updates
  2. Bulk Upload: Enables creation of multiple entities simultaneously through Excel file uploads

The module is built with React, TypeScript, and integrates with a dedicated bulk operations API service for processing large datasets efficiently.

2. Architecture

2.1 High-Level Architecture

graph TB
    subgraph "Frontend Layer"
        UI[React UI Components]
        State[Redux State Management]
        Services[API Service Layer]
    end
    
    subgraph "Backend Layer"
        API[Bulk Operations API]
        Processor[File Processor]
        Validator[Data Validator]
        DB[Database]
    end
    
    UI --> State
    State --> Services
    Services --> API
    API --> Processor
    Processor --> Validator
    Validator --> DB
    
    style UI fill:#61dafb
    style State fill:#764abc
    style Services fill:#f0db4f
    style API fill:#68a063
    style DB fill:#336791

2.2 Module Dependencies

graph LR
    BulkOpsPage[BulkOperationsPage]
    Header[Header Component]
    Title[Title Component]
    MainSection[MainSection]
    
    BulkEdit[BulkEdit Component]
    BulkUpload[BulkUpload Component]
    
    BulkOpsPage --> Header
    BulkOpsPage --> Title
    BulkOpsPage --> MainSection
    
    MainSection --> BulkEdit
    MainSection --> BulkUpload
    
    BulkEdit --> EditServices[Bulk Edit Services]
    BulkUpload --> UploadServices[Bulk Upload Services]
    
    EditServices --> API[Bulk Ops API]
    UploadServices --> API

3. Module Structure

The bulk operations module is organized into the following directory structure:

src/
├── pages/
│   └── bulkOpsRevamp/
│       ├── BulkOperationsPage.tsx      # Main container component
│       ├── Header.tsx                  # Page header with navigation
│       ├── Title.tsx                   # Page title component
│       ├── MainSection.tsx             # Tab container for Edit/Upload
│       ├── BulkEdit.tsx               # Bulk edit functionality
│       ├── BulkUpload.tsx             # Bulk upload functionality
│       ├── bulkOpsUtils.ts            # Utility functions
│       └── hooks/
│           └── useBulkOpsDataOptions.ts # Data options hook
├── components/
│   └── bulkOpsRevamp/
│       ├── BulkEditFilters.tsx        # Filter components
│       ├── StepperConfirmationModal.tsx # Progress stepper
│       ├── OptionsSelector/           # Entity selection
│       ├── TabButtons.tsx             # Tab navigation
│       ├── UploadGuideCard.tsx        # Upload instructions
│       └── UploadInput.tsx            # File upload component
├── services/
│   └── bulkOpsRevamp/
│       ├── bulkUpload.service.ts      # Upload API calls
│       ├── bulkEdit.service.ts        # Edit API calls
│       ├── bulkDownloadQuestionnaire.service.ts
│       └── header.service.ts
└── reducers/
    └── bulkOps/
        ├── bulkOps.reducer.ts         # Redux reducer
        └── bulkOps.action.ts          # Redux actions

4. Component Hierarchy

4.1 Main Page Structure

The BulkOperationsPage serves as the root component, implementing a layout with:

  • Custom styled wrapper with fade-in animation
  • Responsive design with viewport height management
  • Integration with the global layout system
const BulkOperationsPage = () => {
    const pageContainerHeightStyle = usePageWrapperHeightStyle();
    
    useEffect(() => {
        hideBodyElementScrollbar();
    }, []);
    
    return (
        <Layout>
            <AdminPage>
                <PageWrapper customStyles={pageContainerHeightStyle}>
                    <Header />
                    <div className="aa-p-6 aa-h-[91vh]">
                        <Title />
                        <MainSection />
                    </div>
                </PageWrapper>
            </AdminPage>
        </Layout>
    );
};

4.2 Component Details

4.2.1 Header Component

  • Location: src/pages/bulkOpsRevamp/Header.tsx
  • Purpose: Provides navigation breadcrumbs and contextual information
  • Features:
    • Back navigation to admin dashboard
    • Current page indicator
    • Responsive design for mobile/desktop

4.2.2 Title Component

4.2.3 MainSection Component

  • Location: src/pages/bulkOpsRevamp/MainSection.tsx
  • Purpose: Container for tab navigation between Edit and Upload modes
  • Features:
    • Tab-based navigation
    • Conditional rendering based on selected mode
    • State management for active tab

4.2.4 BulkEdit Component

  • Location: src/pages/bulkOpsRevamp/BulkEdit.tsx
  • Purpose: Handles bulk editing workflow
  • Key Features:
    • Entity type selection (Sites, Departments, Users, etc.)
    • Dynamic filter options for specific entities
    • Template download functionality
    • Validation before download
    • Progress tracking

4.2.5 BulkUpload Component

  • Location: src/pages/bulkOpsRevamp/BulkUpload.tsx
  • Purpose: Manages bulk upload workflow
  • Key Features:
    • 4-step guided upload process
    • File validation
    • Real-time status updates
    • Error reporting
    • Success summary

5. Routes and Navigation

5.1 Route Configuration

The bulk operations module is accessible through the admin routes configuration:

Route PathComponentAccess RequiredFeature Flag
/admin/bulk-operationsBulkOperationsPageADMIN_SITES_ALLfeatureBulkOps

Route Definition (src/routes/admin-routes.js:467-473):

<Route
    exact
    path="/admin/bulk-operations"
    component={featureBulkOps ? BulkOperationsPage : null}
    withValidation
    access={RoleResources.ADMIN_SITES_ALL}
/>

5.2 Navigation Flow

graph LR
    Dashboard[Admin Dashboard]
    BulkOps[Bulk Operations Page]
    EditMode[Bulk Edit Mode]
    UploadMode[Bulk Upload Mode]
    
    Dashboard -->|/admin/bulk-operations| BulkOps
    BulkOps --> EditMode
    BulkOps --> UploadMode
    
    EditMode -->|Download Template| ExcelFile1[Excel Template]
    UploadMode -->|Upload File| Processing[File Processing]
    
    Processing -->|Success| Summary[Results Summary]
    Processing -->|Error| ErrorReport[Error Report]

6. API Endpoints

The bulk operations module communicates with a dedicated API service. The base URL is configured through environment variables:

export const bulkOpsUrl = process.env.REACT_APP_BULK_OPS_API_URL;

6.1 Endpoint Details

MethodEndpointPurposeRequest BodyResponse
POST/bulk-operationsUpload bulk operation fileFormData with file and metadata{ bulkOperationsID, status }
GET/bulk-operations/{bulkOperationsID}Check upload status-Status object with progress details
POST/bulk-edit/downloadDownload bulk edit templateEntity type and filtersExcel file stream
GET/bulk-edit/[[Features/Questionnaires/Questionnaires|questionnaires]]Get questionnaire optionsQuery parametersQuestionnaire list
POST/bulk-edit/[[Features/Schedule/Schedule Listing/Schedule Listing|schedules]]Get schedule optionsDate range and filtersSchedule list

6.2 API Service Implementation

6.2.1 Bulk Upload Service

src/services/bulkOpsRevamp/bulkUpload.service.ts

export const uploadBulkOperationFile = async (
    file: File,
    entityType: string,
    additionalData?: any
): Promise<BulkOperationResponse> => {
    const formData = new FormData();
    formData.append('file', file);
    formData.append('entityType', entityType);
    
    if (additionalData) {
        Object.keys(additionalData).forEach(key => {
            formData.append(key, additionalData[key]);
        });
    }
    
    const response = await fetch(`${bulkOpsUrl}/bulk-operations`, {
        method: 'POST',
        headers: {
            'Authorization': `Bearer ${getAuthToken()}`
        },
        body: formData
    });
    
    return response.json();
};

6.2.2 Status Checking Service

export const checkBulkOperationStatus = async (
    bulkOperationsID: string
): Promise<BulkOperationStatus> => {
    const response = await fetch(
        `${bulkOpsUrl}/bulk-operations/${bulkOperationsID}`,
        {
            headers: {
                'Authorization': `Bearer ${getAuthToken()}`
            }
        }
    );
    
    return response.json();
};

7. State Management

The bulk operations module uses Redux for state management, specifically for managing filter states:

7.1 Redux State Structure

src/reducers/bulkOps/bulkOps.reducer.ts

interface BulkOpsState {
    openQuestionnaireBulkEditFilter: boolean;
    openSchedulesBulkEditFilter: boolean;
}
 
const initialState: BulkOpsState = {
    openQuestionnaireBulkEditFilter: false,
    openSchedulesBulkEditFilter: false
};

7.2 Actions

src/reducers/bulkOps/bulkOps.action.ts

export const TOGGLE_QUESTIONNAIRE_BULK_EDIT_FILTER = 'TOGGLE_QUESTIONNAIRE_BULK_EDIT_FILTER';
export const TOGGLE_SCHEDULES_BULK_EDIT_FILTER = 'TOGGLE_SCHEDULES_BULK_EDIT_FILTER';
 
export const toggleQuestionnaireBulkEditFilter = () => ({
    type: TOGGLE_QUESTIONNAIRE_BULK_EDIT_FILTER
});
 
export const toggleSchedulesBulkEditFilter = () => ({
    type: TOGGLE_SCHEDULES_BULK_EDIT_FILTER
});

7.3 Local Component State

Individual components manage their own state for:

  • Selected entity type
  • Upload progress
  • File selection
  • Error messages
  • Success indicators

8. Business Logic and Workflows

8.1 Bulk Edit Workflow

The bulk edit process follows a specific workflow designed to ensure data integrity:

stateDiagram-v2
    [*] --> EntitySelection
    EntitySelection --> FilterConfiguration: Select entity with filters
    EntitySelection --> DirectDownload: Select entity without filters
    
    FilterConfiguration --> TemplateGeneration
    DirectDownload --> TemplateGeneration
    
    TemplateGeneration --> Downloading
    Downloading --> ExcelEditing: User downloads file
    
    ExcelEditing --> FileUpload: User edits offline
    FileUpload --> Validation
    
    Validation --> Processing: Valid data
    Validation --> ErrorReport: Invalid data
    
    Processing --> UpdateDatabase
    UpdateDatabase --> SuccessSummary
    
    ErrorReport --> ExcelEditing: Fix errors
    SuccessSummary --> [*]

8.1.1 Entity Selection Logic

The system supports multiple entity types, each with specific configuration requirements:

  1. Sites

    • No additional filters required
    • Downloads all sites for the organization
    • Includes site hierarchy information

  2. Departments

    • No additional filters required
    • Downloads all departments across sites
    • Maintains site-department relationships

  3. Users

    • No additional filters required
    • Downloads all users with role information
    • Includes site assignments

  4. Questionnaires

    • Requires filter selection
    • Options: Active, Inactive, or specific questionnaires
    • Includes question structure and scoring

  5. Off Days

    • No additional filters required
    • Downloads all off-day configurations
    • Includes recurrence patterns

  6. Schedules

    • Requires date range selection
    • Optional site/department filters
    • Downloads schedules within specified period

8.1.2 Filter Logic Implementation

For entities requiring filters, the system implements a multi-step process:

const handleEntitySelection = (entity: EntityType) => {
    if (requiresFilter(entity)) {
        openFilterModal(entity);
    } else {
        proceedToDownload(entity);
    }
};
 
const requiresFilter = (entity: EntityType): boolean => {
    return ['questionnaires', 'schedules'].includes(entity);
};

8.2 Bulk Upload Workflow

The bulk upload process is designed as a 4-step wizard:

graph TD
    Start[Start Upload Process]
    Step1[Step 1: Select File]
    Step2[Step 2: Validate Data]
    Step3[Step 3: Process Entities]
    Step4[Step 4: View Results]
    
    Start --> Step1
    Step1 -->|File Selected| Step2
    Step2 -->|Valid| Step3
    Step2 -->|Invalid| ErrorDisplay[Display Errors]
    ErrorDisplay --> Step1
    Step3 -->|Processing| StatusCheck[Check Status]
    StatusCheck -->|In Progress| StatusCheck
    StatusCheck -->|Complete| Step4
    Step4 --> End[End Process]

8.2.1 Upload Processing Logic

  1. File Validation

    • File type check (.xlsx only)
    • File size validation (max 10MB)
    • Initial format verification

  2. Data Validation

    • Column header verification
    • Required field validation
    • Data type checking
    • Business rule validation

  3. Entity Creation

    • Batch processing for performance
    • Transaction management
    • Rollback on failure
    • Progress tracking

  4. Result Summary

    • Success count
    • Error details
    • Created entity IDs
    • Downloadable report

8.3 Status Polling Implementation

The system implements intelligent polling for tracking upload progress:

const pollStatus = async (bulkOperationsID: string) => {
    const interval = setInterval(async () => {
        const status = await checkBulkOperationStatus(bulkOperationsID);
        
        updateProgress(status);
        
        if (status.status === 'COMPLETED' || status.status === 'FAILED') {
            clearInterval(interval);
            handleCompletion(status);
        }
    }, 2000); // Poll every 2 seconds
};

8.4 Error Handling Strategies

The module implements comprehensive error handling:

  1. Client-Side Validation

    • Immediate feedback for file format issues
    • Size limit enforcement
    • Required field validation

  2. Server-Side Validation

    • Business rule validation
    • Data integrity checks
    • Relationship validation

  3. Error Reporting

    • Row-level error details
    • Downloadable error report
    • Suggested fixes
    • Error categorization

9. Data Models and Types

9.1 Entity Type Definitions

export enum EntityType {
    SITES = 'sites',
    DEPARTMENTS = 'departments',
    USERS = 'users',
    QUESTIONNAIRES = 'questionnaires',
    OFF_DAYS = 'offDays',
    SCHEDULES = 'schedules'
}
 
export interface BulkOperationFile {
    id: string;
    filename: string;
    entityType: EntityType;
    status: OperationStatus;
    createdAt: Date;
    updatedAt: Date;
    createdBy: string;
    totalRecords: number;
    processedRecords: number;
    successCount: number;
    errorCount: number;
}
 
export enum OperationStatus {
    PENDING = 'PENDING',
    VALIDATING = 'VALIDATING',
    PROCESSING = 'PROCESSING',
    COMPLETED = 'COMPLETED',
    FAILED = 'FAILED',
    PARTIAL_SUCCESS = 'PARTIAL_SUCCESS'
}

9.2 Filter Models

export interface QuestionnaireFilter {
    status?: 'active' | 'inactive' | 'all';
    questionnaireIds?: string[];
    includeArchived?: boolean;
}
 
export interface ScheduleFilter {
    dateFrom: Date;
    dateTo: Date;
    siteIds?: string[];
    departmentIds?: string[];
    assigneeIds?: string[];
}

9.3 Response Models

export interface BulkOperationResponse {
    bulkOperationsID: string;
    status: OperationStatus;
    message?: string;
    errors?: ValidationError[];
}
 
export interface ValidationError {
    row: number;
    column: string;
    value: any;
    error: string;
    suggestion?: string;
}
 
export interface ProgressUpdate {
    totalRecords: number;
    processedRecords: number;
    successCount: number;
    errorCount: number;
    currentStep: string;
    estimatedTimeRemaining?: number;
}

10. Package Dependencies

The bulk operations module relies on several key packages:

PackageVersionPurposeUsage in Module
react^18.2.0Core frameworkComponent development
react-redux^8.0.5State managementFilter state management
styled-components^5.3.6CSS-in-JS stylingComponent styling, animations
@tanstack/react-query^4.29.5Server state managementAPI call caching
react-hook-form^7.43.9Form managementFilter forms
xlsx^0.18.5Excel file handlingClient-side file preview
date-fns^2.29.3Date manipulationSchedule date handling
react-toastify^9.1.2Toast notificationsSuccess/error messages
react-dropzone^14.2.3File upload UIDrag-and-drop upload
@mui/material^5.11.16UI componentsModals, buttons, inputs

10.1 Package-Specific Implementations

10.1.1 Styled Components Usage

The module extensively uses styled-components for creating reusable, themed components:

const PageWrapper = styled.div<PageWrapperProps>`
    position: relative;
    width: 100%;
    max-height: 100vh;
    animation: 150ms ${fadeIn} ease-out;
    overflow-y: auto;
    
    @media (min-width: 992px) {
        background: #fff;
    }
    
    ${(props) => props.customStyles ?? ''};
`;

10.1.2 React Query Integration

API calls are managed through React Query for optimal caching and synchronization:

const useBulkOperationStatus = (bulkOperationsID: string) => {
    return useQuery({
        queryKey: ['bulkOperationStatus', bulkOperationsID],
        queryFn: () => checkBulkOperationStatus(bulkOperationsID),
        refetchInterval: 2000,
        enabled: !!bulkOperationsID
    });
};

11. Error Handling

11.1 Error Categories

  1. File Errors

    • Invalid file format
    • File too large
    • Corrupted file
    • Missing required sheets

  2. Validation Errors

    • Missing required fields
    • Invalid data types
    • Business rule violations
    • Referential integrity errors

  3. Processing Errors

    • Server timeout
    • Database constraints
    • Concurrent modification
    • Permission denied

  4. Network Errors

    • Connection timeout
    • Service unavailable
    • Authentication failure

11.2 Error Response Format

interface ErrorResponse {
    code: string;
    message: string;
    details?: {
        field?: string;
        row?: number;
        value?: any;
        suggestion?: string;
    }[];
    timestamp: string;
    requestId: string;
}

11.3 User-Friendly Error Messages

The system translates technical errors into actionable user messages:

const errorMessageMap: Record<string, string> = {
    'FILE_TOO_LARGE': 'File size exceeds 10MB limit. Please split into smaller files.',
    'INVALID_FORMAT': 'File must be in .xlsx format. Please save as Excel file.',
    'DUPLICATE_ENTRY': 'Duplicate entries found. Check highlighted rows.',
    'MISSING_REQUIRED': 'Required fields are missing. See error report for details.',
    'PERMISSION_DENIED': 'You do not have permission for this operation.'
};

12. Security Considerations

12.1 Authentication & Authorization

  1. Token-Based Authentication

    • JWT tokens for API requests
    • Automatic token refresh
    • Secure token storage

  2. Role-Based Access Control

    • ADMIN_SITES_ALL permission required
    • Entity-level permissions
    • Operation-level restrictions

  3. Data Security

    • HTTPS for all communications
    • File encryption in transit
    • Temporary file cleanup
    • Audit logging

12.2 Input Validation

const validateFileUpload = (file: File): ValidationResult => {
    // File type validation
    if (!file.name.endsWith('.xlsx')) {
        return { valid: false, error: 'Invalid file type' };
    }
    
    // File size validation
    if (file.size > 10 * 1024 * 1024) { // 10MB
        return { valid: false, error: 'File too large' };
    }
    
    // Additional security checks
    if (containsMacros(file)) {
        return { valid: false, error: 'Macros not allowed' };
    }
    
    return { valid: true };
};

13. Performance Optimizations

13.1 Frontend Optimizations

  1. Code Splitting

    • Lazy loading of bulk operations module
    • Dynamic imports for large components
    • Route-based code splitting

  2. Memoization

    • React.memo for pure components
    • useMemo for expensive calculations
    • useCallback for event handlers

  3. Virtual Scrolling

    • Large data set rendering
    • Pagination for result display
    • Incremental loading

13.2 API Optimizations

  1. Batch Processing

    • Group operations by type
    • Bulk database operations
    • Optimized query execution

  2. Caching Strategy

    • Template caching
    • Filter option caching
    • Result caching

  3. Streaming

    • Large file downloads
    • Progress updates
    • Partial results

13.3 Example Performance Implementation

const BulkEditFilters = React.memo(({ 
    entityType, 
    onFilterApply 
}: FilterProps) => {
    const filterOptions = useMemo(() => 
        generateFilterOptions(entityType), 
        [entityType]
    );
    
    const handleApply = useCallback((filters) => {
        onFilterApply(filters);
    }, [onFilterApply]);
    
    return (
        <FilterComponent 
            options={filterOptions}
            onApply={handleApply}
        />
    );
});

14. Monitoring and Analytics

14.1 Performance Metrics

  1. Upload Metrics

    • Average upload time
    • Success rate
    • Error frequency
    • File size distribution

  2. User Analytics

    • Feature usage
    • Common error patterns
    • User journey tracking
    • Abandonment rate

  3. System Metrics

    • API response times
    • Database performance
    • Memory usage
    • CPU utilization

14.2 Logging Strategy

const logBulkOperation = (operation: BulkOperation) => {
    logger.info('Bulk operation initiated', {
        userId: operation.userId,
        entityType: operation.entityType,
        recordCount: operation.recordCount,
        timestamp: new Date().toISOString()
    });
};

15. User Interface Components

15.1 Component Library

The bulk operations module uses a combination of custom components and shared UI elements:

15.1.1 Tab Navigation System

TabButtons Component

interface TabButtonsProps {
    tabs: Array<{
        label: string;
        value: string;
    }>;
    activeTab: string;
    onTabChange: (tab: string) => void;
}

The tab system uses:

  • Active state: Amber underline (#FCD34D)
  • Active text: 25282B
  • Inactive text: 8F8F8F
  • Smooth transitions on hover/click

15.1.2 Entity Selection Interface

OptionsSelector Component

Provides checkbox-based selection with:

  • Custom checkbox styling
  • Filter indicators for entities requiring configuration
  • Purple accent color (#574FCF)
  • Responsive grid layout

15.1.3 File Upload Components

15.1.3.1 UploadInput

src/components/bulkOpsRevamp/UploadInput.tsx

  • Drag-and-drop support
  • File type validation (.xlsx only)
  • Visual feedback for file selection
  • Delete functionality with confirmation
15.1.3.2 UploadGuideCard

src/components/bulkOpsRevamp/UploadGuideCard.tsx

  • Step-by-step visual guide
  • Numbered steps with descriptions
  • Consistent styling with gray background (#F9F8F8)

15.1.4 Progress Indicators

15.1.4.1 Step Component

src/components/bulkOpsRevamp/Step.tsx

interface StepProps {
    state: 'complete' | 'pending' | 'error' | 'in-progress';
    stepNumber: number;
    label: string;
}

Visual states:

  • Complete: Purple checkmark (#574FCF)
  • In Progress: Purple radio button
  • Pending: Gray circle (#E2E2E2)
  • Error: Red error icon (#F6001A)

15.2 Design System Integration

15.2.1 Color Palette

const bulkOpsColors = {
    primary: {
        purple: '#574FCF',
        purpleHover: '#4741B3'
    },
    status: {
        success: '#56C768',
        error: '#F6001A',
        warning: '#FFA500',
        info: '#3B82F6'
    },
    neutral: {
        text: '#25282B',
        textSecondary: '#8F8F8F',
        border: '#C4C4C4',
        background: '#F9F8F8',
        backgroundAlt: '#EFEEED'
    }
};

15.2.2 Typography

/* Base font family */
.aa-font-sans {
    font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
}
 
/* Text sizes */
.aa-text-sm { font-size: 0.875rem; }
.aa-text-base { font-size: 1rem; }
.aa-text-md { font-size: 1.125rem; }
.aa-text-lg { font-size: 1.25rem; }
 
/* Font weights */
.aa-font-normal { font-weight: 400; }
.aa-font-semibold { font-weight: 600; }

15.3 Modal System

15.3.1 StepperConfirmationModal

src/components/bulkOpsRevamp/StepperConfirmationModal.tsx

The modal system provides:

  • Full-screen overlay
  • Centered content with max-width constraints
  • Progress stepper integration
  • Timer display for long operations
  • Error log viewer with CSV export
  • Success summary display

15.4 Form Controls

15.4.1 Filter Components

  1. BulkEditFilters

    • Complex filter system for questionnaires and schedules
    • Date range picker integration
    • Multi-select dropdowns with search
    • Required field indicators

  2. CustomizedDateRangePicker

    • Wraps calendar component
    • Preset date ranges
    • Custom date format (D MMM YYYY)
    • Responsive modal overlay

15.5 Responsive Design

The module implements responsive design principles:

// Mobile-first breakpoints
const breakpoints = {
    sm: '640px',
    md: '768px',
    lg: '1024px',
    xl: '1280px'
};
 
// Example responsive styling
const ResponsiveContainer = styled.div`
    padding: 1rem;
    
    @media (min-width: ${breakpoints.md}) {
        padding: 1.5rem;
    }
    
    @media (min-width: ${breakpoints.lg}) {
        padding: 2rem;
        max-width: 1200px;
        margin: 0 auto;
    }
`;

16. Internationalization (i18n)

16.1 Language Support

The bulk operations module is fully internationalized using react-i18next:

16.1.1 Translation Keys Structure

{
    "label": {
        "bulkOpsRevamp": {
            "bulkOperations": "Bulk Operations",
            "downloadTemplate": "Download Template",
            "selectFile": "Select File",
            "upload": "Upload",
            "processing": "Processing...",
            "completed": "Completed",
            "failed": "Failed",
            "filters": {
                "questionnaire": "Questionnaire",
                "dateRange": "Date Range",
                "sites": "Sites",
                "departments": "Departments",
                "auditors": "Auditors"
            }
        }
    },
    "message": {
        "bulkOpsRevamp": {
            "mainTitle": {
                "title": "Streamline Data Management",
                "description": "Efficiently manage large datasets through bulk operations"
            },
            "bulkEdit": {
                "selectEntity": "Select entities to download for editing",
                "applyFilters": "Apply filters to narrow down your selection",
                "downloadInProgress": "Preparing your download...",
                "downloadComplete": "Download completed successfully"
            },
            "bulkUpload": {
                "dragDropFile": "Drag and drop your file here or click to browse",
                "fileValidation": "Validating file format and contents...",
                "uploadProgress": "Upload in progress: {progress}%",
                "validationErrors": "Validation errors found. Please review and correct."
            }
        }
    }
}

16.2 Usage in Components

import { useTranslation } from 'react-i18next';
 
const BulkOperationsTitle = () => {
    const { t } = useTranslation();
    
    return (
        <div>
            <h1>{t('label.bulkOpsRevamp.bulkOperations')}</h1>
            <p>{t('message.bulkOpsRevamp.mainTitle.description')}</p>
        </div>
    );
};

17. Detailed Component Workflows

17.1 Bulk Edit Download Flow

sequenceDiagram
    participant User
    participant UI as BulkEdit Component
    participant Filter as Filter Modal
    participant Service as API Service
    participant Backend as Bulk Ops API
    
    User->>UI: Select entity type
    
    alt Entity requires filters
        UI->>Filter: Open filter modal
        User->>Filter: Configure filters
        Filter->>UI: Return filter config
    end
    
    UI->>UI: Show download button
    User->>UI: Click download
    
    UI->>Service: Request template download
    Service->>Service: Exchange Firebase token
    Service->>Backend: POST /bulk-edit/download
    
    Backend->>Backend: Generate template
    Backend->>Service: Return Excel file
    Service->>UI: Trigger file download
    UI->>User: File downloaded

17.2 Bulk Upload Processing Flow

sequenceDiagram
    participant User
    participant UI as BulkUpload Component
    participant Service as Upload Service
    participant Backend as Bulk Ops API
    participant Poller as Status Poller
    
    User->>UI: Select .xlsx file
    UI->>UI: Validate file locally
    
    UI->>Service: Upload file
    Service->>Backend: POST /bulk-operations
    Backend->>Service: Return operationId
    Service->>UI: Upload initiated
    
    UI->>Poller: Start polling
    
    loop Every 2 seconds
        Poller->>Backend: GET /bulk-operations/{id}
        Backend->>Poller: Return status
        Poller->>UI: Update progress
    end
    
    alt Success
        UI->>User: Show success summary
    else Validation Error
        UI->>User: Show error log
        User->>UI: Download error CSV
    end

18. Advanced Configuration

18.1 Entity Type Configuration

The bulk operations module supports configurable entity types defined in bulkOpsUtils.ts:

export const BULK_OPS_OPTIONS = [
    {
        label: 'Sites',
        value: 'sites',
        hasFilter: false,
        requiredPermission: 'ADMIN_SITES_ALL',
        maxRecords: 10000
    },
    {
        label: 'Departments',
        value: 'departments',
        hasFilter: false,
        requiredPermission: 'ADMIN_DEPARTMENTS_ALL',
        maxRecords: 50000
    },
    {
        label: 'Users',
        value: 'users',
        hasFilter: false,
        requiredPermission: 'ADMIN_USERS_ALL',
        maxRecords: 25000
    },
    {
        label: 'Questionnaires',
        value: 'questionnaires',
        hasFilter: true,
        filterType: 'questionnaire',
        requiredPermission: 'ADMIN_QUESTIONNAIRES_ALL',
        maxRecords: 1000
    },
    {
        label: 'Off Days',
        value: 'offDays',
        hasFilter: false,
        requiredPermission: 'ADMIN_SCHEDULES_ALL',
        maxRecords: 5000
    },
    {
        label: 'Schedules',
        value: 'schedules',
        hasFilter: true,
        filterType: 'schedule',
        requiredPermission: 'ADMIN_SCHEDULES_ALL',
        maxRecords: 100000
    }
];

18.2 Upload Step Configuration

export const UPLOAD_GUIDE_STEPS = [
    {
        step: 1,
        title: 'Download Template',
        description: 'Get the Excel template for your entity type',
        icon: 'download'
    },
    {
        step: 2,
        title: 'Fill Data',
        description: 'Enter your data following the template format',
        icon: 'edit'
    },
    {
        step: 3,
        title: 'Upload File',
        description: 'Upload the completed Excel file',
        icon: 'upload'
    },
    {
        step: 4,
        title: 'Review Results',
        description: 'Check the upload summary and any errors',
        icon: 'check'
    }
];

18.3 Stepper State Configuration

export const STEPPER_STATES = {
    BULK_EDIT: {
        IDLE: { steps: [] },
        DOWNLOADING: {
            steps: [
                { label: 'Preparing download', state: 'in-progress' },
                { label: 'Generating template', state: 'pending' },
                { label: 'Download complete', state: 'pending' }
            ]
        },
        COMPLETE: {
            steps: [
                { label: 'Preparing download', state: 'complete' },
                { label: 'Generating template', state: 'complete' },
                { label: 'Download complete', state: 'complete' }
            ]
        }
    },
    BULK_UPLOAD: {
        UPLOADING: {
            steps: [
                { label: 'Uploading file', state: 'in-progress' },
                { label: 'Validating data', state: 'pending' },
                { label: 'Creating entities', state: 'pending' },
                { label: 'Complete', state: 'pending' }
            ]
        },
        VALIDATING: {
            steps: [
                { label: 'Uploading file', state: 'complete' },
                { label: 'Validating data', state: 'in-progress' },
                { label: 'Creating entities', state: 'pending' },
                { label: 'Complete', state: 'pending' }
            ]
        },
        PROCESSING: {
            steps: [
                { label: 'Uploading file', state: 'complete' },
                { label: 'Validating data', state: 'complete' },
                { label: 'Creating entities', state: 'in-progress' },
                { label: 'Complete', state: 'pending' }
            ]
        },
        COMPLETE: {
            steps: [
                { label: 'Uploading file', state: 'complete' },
                { label: 'Validating data', state: 'complete' },
                { label: 'Creating entities', state: 'complete' },
                { label: 'Complete', state: 'complete' }
            ]
        },
        ERROR: {
            steps: [
                { label: 'Uploading file', state: 'complete' },
                { label: 'Validating data', state: 'error' },
                { label: 'Creating entities', state: 'pending' },
                { label: 'Complete', state: 'pending' }
            ]
        }
    }
};

19. Testing Strategy

19.1 Unit Testing

The bulk operations module uses Jest and React Testing Library:

// Example test for BulkUpload component
import { render, screen, fireEvent, waitFor } from '@testing-library/react';
import { BulkUpload } from '../BulkUpload';
 
describe('BulkUpload', () => {
    it('should accept only .xlsx files', async () => {
        render(<BulkUpload />);
        
        const file = new File(['content'], 'test.pdf', { type: 'application/pdf' });
        const input = screen.getByLabelText('upload-input');
        
        fireEvent.change(input, { target: { files: [file] } });
        
        await waitFor(() => {
            expect(screen.getByText('File must be .xlsx format')).toBeInTheDocument();
        });
    });
    
    it('should show upload progress', async () => {
        render(<BulkUpload />);
        
        const file = new File(['content'], 'test.xlsx', { 
            type: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet' 
        });
        
        const input = screen.getByLabelText('upload-input');
        fireEvent.change(input, { target: { files: [file] } });
        
        await waitFor(() => {
            expect(screen.getByText(/Uploading file/)).toBeInTheDocument();
        });
    });
});

19.2 Integration Testing

// Example integration test
import { renderWithProviders } from 'test-utils';
import { BulkOperationsPage } from '../BulkOperationsPage';
import { server } from 'mocks/server';
 
describe('BulkOperationsPage Integration', () => {
    beforeAll(() => server.listen());
    afterEach(() => server.resetHandlers());
    afterAll(() => server.close());
    
    it('should complete full upload workflow', async () => {
        const { user } = renderWithProviders(<BulkOperationsPage />);
        
        // Switch to upload tab
        await user.click(screen.getByText('Upload'));
        
        // Upload file
        const file = createMockExcelFile();
        const input = screen.getByLabelText('upload-input');
        await user.upload(input, file);
        
        // Wait for processing
        await waitFor(() => {
            expect(screen.getByText('Upload completed')).toBeInTheDocument();
        }, { timeout: 10000 });
        
        // Verify success summary
        expect(screen.getByText(/Created: \d+ entities/)).toBeInTheDocument();
    });
});

20. Troubleshooting Guide

20.1 Common Issues

20.1.1 File Upload Failures

Problem: File upload fails with “Network Error”

Solutions:

  • Check file size (max 10MB)
  • Verify file format is .xlsx
  • Ensure stable internet connection
  • Check API service availability

20.1.2 Template Download Issues

Problem: Template download doesn’t start

Solutions:

  • Check browser popup blocker
  • Verify authentication token
  • Clear browser cache
  • Try different browser

20.1.3 Validation Errors

Problem: Consistent validation errors on upload

Solutions:

  • Download fresh template
  • Check date formats (DD/MM/YYYY)
  • Verify required fields are filled
  • Remove special characters from text fields

20.1.4 Performance Issues

Problem: Slow upload/download for large files

Solutions:

  • Split large datasets into smaller batches
  • Upload during off-peak hours
  • Check network bandwidth
  • Contact support for bulk limits increase

20.2 Debug Mode

Enable debug mode for detailed logging:

// In browser console
localStorage.setItem('BULK_OPS_DEBUG', 'true');
 
// This enables:
// - Detailed API request/response logging
// - Component lifecycle logging
// - Performance metrics
// - Error stack traces

21. API Rate Limiting

The bulk operations API implements rate limiting:

OperationLimitWindow
File Upload10 requests5 minutes
Status Check100 requests1 minute
Template Download20 requests10 minutes

Exceeding limits returns:

{
    "error": "RATE_LIMIT_EXCEEDED",
    "message": "Too many requests. Please try again later.",
    "retryAfter": 300
}

22. Migration Guide

22.1 Migrating from Legacy Bulk Operations

For users transitioning from the old bulk operations system:

22.1.1 Key Differences

  1. File Format

    • Old: CSV format
    • New: Excel (.xlsx) format only

  2. Entity Support

    • Old: Sites, Users only
    • New: All 6 entity types

  3. Error Handling

    • Old: Email notification
    • New: Real-time error display with downloadable logs

  4. Progress Tracking

    • Old: No progress indication
    • New: Step-by-step progress with time estimates

22.1.2 Migration Steps

  1. Export existing data using old system
  2. Convert CSV to Excel format
  3. Download new templates to ensure column compatibility
  4. Map old columns to new format
  5. Test with small batch before full migration
  6. Validate results using error logs

23. Documentation Summary

This comprehensive technical documentation covers all aspects of the Bulk Operations module:

✅ Module Structure - Complete file organization and component hierarchy
✅ API Integration - Detailed endpoint specifications and authentication
✅ UI Components - Comprehensive component library documentation
✅ Business Logic - Workflow diagrams and processing logic
✅ Error Handling - Robust error management strategies
✅ Security - Authentication, validation, and security measures
✅ Performance - Optimization techniques and monitoring
✅ Testing - Testing strategies and implementation
✅ Troubleshooting - Common issues and solutions

23.1 Quick Reference

23.1.1 Essential Files

23.1.2 Key API Endpoints

  • Upload: POST /bulk-operations
  • Status: GET /bulk-operations/{id}
  • Download: POST /bulk-edit/download
  • Auth: GET /[[Features/Authentication/Authentication|auth]]/retrieve-jwt

23.1.3 Supported Entity Types

  1. Sites - No filters required
  2. Departments - No filters required
  3. Users - No filters required
  4. Questionnaires - Filter options available
  5. Off Days - No filters required
  6. Schedules - Date range and filter options

23.1.4 Environment Configuration

REACT_APP_BULK_OPS_API_URL=https://api.nimbly.com/bulk-ops
REACT_APP_CLOUD_API_URL=https://api.nimbly.com/cloud

Graph View