Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat : added target sum mapping #825

Merged
merged 2 commits into from
Jun 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 64 additions & 0 deletions utilities/project-factory/src/server/api/genericApis.ts
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,69 @@ const getTargetSheetData = async (
return workbookData;
};

const getTargetSheetDataAfterCode = async (
fileUrl: string,
getRow = false,
getSheetName = false,
codeColumnName = "Boundary Code",
localizationMap?: any
) => {
const workbook = await getTargetWorkbook(fileUrl, localizationMap);
const sheetNames: string[] = [];
workbook.eachSheet((worksheet: any) => {
sheetNames.push(worksheet.name);
});
const localizedSheetNames = getLocalizedHeaders(sheetNames, localizationMap);

const workbookData: { [key: string]: any[] } = {}; // Object to store data from each sheet

for (const sheetName of localizedSheetNames) {
const worksheet = workbook.getWorksheet(sheetName);
const sheetData = worksheet.getSheetValues({ includeEmpty: true });

// Find the target column index where the first row value matches codeColumnName
const firstRow = sheetData[1];
let targetColumnIndex = -1;
for (let colIndex = 1; colIndex < firstRow.length; colIndex++) {
if (firstRow[colIndex] === codeColumnName) {
targetColumnIndex = colIndex;
break;
}
}

if (targetColumnIndex === -1) {
console.warn(`Column "${codeColumnName}" not found in sheet "${sheetName}".`);
continue;
}

// Process data from sheet
const processedData = sheetData.map((row: any, rowIndex: any) => {
if (rowIndex === 0) return null; // Skip header row

let rowData = getJsonData([row], getRow, getSheetName, sheetName)[0];
if (!rowData) return null;

// Add integer values in the target column for the current row
let sum = 0;
for (let colIndex = targetColumnIndex + 1; colIndex < row.length; colIndex++) {
const value = row[colIndex];
if (typeof value === 'number' && Number.isInteger(value)) {
sum += value;
}
}

// Add the sum to the row data
rowData['Target at the Selected Boundary level'] = sum;
return rowData;
}).filter(Boolean); // Remove null entries

workbookData[sheetName] = processedData;
}

return workbookData;
};
Comment on lines +141 to +201
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add comprehensive error handling in getTargetSheetDataAfterCode
The function getTargetSheetDataAfterCode lacks comprehensive error handling. While there is a console warning when a column is not found, it would be beneficial to handle potential errors that could arise from asynchronous operations such as file reading or data processing. Consider adding try-catch blocks around these operations to ensure that any exceptions are caught and handled gracefully.



// Function to search MDMS for specific unique identifiers
const searchMDMS: any = async (
uniqueIdentifiers: any[],
Expand Down Expand Up @@ -1111,6 +1174,7 @@ export {
generateHierarchyList,
getTargetWorkbook,
getTargetSheetData,
getTargetSheetDataAfterCode,
callMdmsData,
getMDMSV1Data,
callMdmsV2Data,
Expand Down
6 changes: 3 additions & 3 deletions utilities/project-factory/src/server/utils/campaignUtils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import config from "../config/index";
import { v4 as uuidv4 } from 'uuid';
import { produceModifiedMessages } from '../kafka/Listener'
import { confirmProjectParentCreation, createProjectCampaignResourcData, getCampaignSearchResponse, getHierarchy, handleResouceDetailsError, projectCreate } from "../api/campaignApis";
import { getCampaignNumber, createAndUploadFile, getSheetData, createExcelSheet, getAutoGeneratedBoundaryCodesHandler, getTargetSheetData, createBoundaryEntities, createBoundaryRelationship, getMDMSV1Data, callMdmsV2Data } from "../api/genericApis";
import { getCampaignNumber, createAndUploadFile, getSheetData, createExcelSheet, getAutoGeneratedBoundaryCodesHandler, createBoundaryEntities, createBoundaryRelationship, getMDMSV1Data, callMdmsV2Data, getTargetSheetDataAfterCode } from "../api/genericApis";
import { getFormattedStringForDebug, logger } from "./logger";
import createAndSearch from "../config/createAndSearch";
import { addDataToSheet, changeFirstRowColumnColour, createBoundaryDataMainSheet, createReadMeSheet, findMapValue, getBoundaryRelationshipData, getConfigurableColumnHeadersFromSchemaForTargetSheet, getLocalizedHeaders, getLocalizedMessagesHandler, modifyBoundaryData, replicateRequest, throwError } from "./genericUtils";
Expand Down Expand Up @@ -1231,9 +1231,9 @@ async function getCodesTarget(request: any, localizationMap?: any) {
if (!fileResponse?.fileStoreIds?.[0]?.url) {
throwError("FILE", 500, "DOWNLOAD_URL_NOT_FOUND");
}
const targetData = await getTargetSheetData(fileResponse?.fileStoreIds?.[0]?.url, true, true);
const boundaryTargetMapping: any = {};
const codeColumnName = getLocalizedName(createAndSearch?.boundaryWithTarget?.boundaryValidation?.column, localizationMap)
const targetData = await getTargetSheetDataAfterCode(fileResponse?.fileStoreIds?.[0]?.url, true, true, codeColumnName);
const boundaryTargetMapping: any = {};
Comment on lines +1235 to +1236
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid using the delete operator for performance reasons. Consider setting the property to undefined instead.

- delete request.body.ExistingCampaignDetails;
+ request.body.ExistingCampaignDetails = undefined;

Committable suggestion was skipped due to low confidence.


Ensure proper error handling for potential undefined values when using optional chaining.

- const targetData = await getTargetSheetDataAfterCode(fileResponse?.fileStoreIds?.[0]?.url, true, true, codeColumnName);
+ if (!fileResponse?.fileStoreIds?.[0]?.url) {
+     throw new Error("URL is undefined");
+ }
+ const targetData = await getTargetSheetDataAfterCode(fileResponse.fileStoreIds[0].url, true, true, codeColumnName);
Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const targetData = await getTargetSheetDataAfterCode(fileResponse?.fileStoreIds?.[0]?.url, true, true, codeColumnName);
const boundaryTargetMapping: any = {};
if (!fileResponse?.fileStoreIds?.[0]?.url) {
throw new Error("URL is undefined");
}
const targetData = await getTargetSheetDataAfterCode(fileResponse.fileStoreIds[0].url, true, true, codeColumnName);
const boundaryTargetMapping: any = {};

// Iterate through each key in targetData
for (const key in targetData) {
// Iterate through each entry in the array under the current key
Expand Down
10 changes: 9 additions & 1 deletion utilities/project-factory/src/server/utils/genericUtils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import { getLocaleFromRequest, getLocalisationModuleName } from "./localisationU
import { getBoundaryColumnName, getBoundaryTabName } from "./boundaryUtils";
import { getBoundaryDataService } from "../service/dataManageService";
import { addDataToSheet, formatWorksheet, getNewExcelWorkbook } from "./excelUtils";
import createAndSearch from "../config/createAndSearch";
const NodeCache = require("node-cache");

const updateGeneratedResourceTopic = config?.kafka?.KAFKA_UPDATE_GENERATED_RESOURCE_DETAILS_TOPIC;
Expand Down Expand Up @@ -562,10 +563,16 @@ function changeFirstRowColumnColour(facilitySheet: any, color: any, columnNumber
firstHeaderCell.fill = {
type: 'pattern',
pattern: 'solid',
fgColor: { argb: color } // Orange color
fgColor: { argb: color }
};
}

function hideUniqueIdentifierColumn(sheet: any, column: any) {
if (column) {
sheet.getColumn(column).hidden = true
}
}
Comment on lines +570 to +574
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ensure that the hideUniqueIdentifierColumn function handles potential errors gracefully.

-  if (column) {
+  if (column && sheet.getColumn(column)) {
     sheet.getColumn(column).hidden = true
   }

This change ensures that the function checks whether the column exists before attempting to hide it, preventing runtime errors if the column is not present.

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
function hideUniqueIdentifierColumn(sheet: any, column: any) {
if (column) {
sheet.getColumn(column).hidden = true
}
}
function hideUniqueIdentifierColumn(sheet: any, column: any) {
if (column && sheet.getColumn(column)) {
sheet.getColumn(column).hidden = true
}
}



async function createFacilityAndBoundaryFile(facilitySheetData: any, boundarySheetData: any, request: any, localizationMap?: any) {
const workbook = getNewExcelWorkbook();
Expand All @@ -582,6 +589,7 @@ async function createFacilityAndBoundaryFile(facilitySheetData: any, boundaryShe
// Add facility sheet data
const facilitySheet = workbook.addWorksheet(localizedFacilityTab);
addDataToSheet(facilitySheet, facilitySheetData, undefined, undefined, true);
hideUniqueIdentifierColumn(facilitySheet, createAndSearch?.["facility"]?.uniqueIdentifierColumn);
changeFirstRowColumnColour(facilitySheet, 'E06666');

// Add boundary sheet to the workbook
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -330,6 +330,16 @@ async function validateViaSchema(data: any, schema: any, request: any, localizat
}
if (data?.length > 0) {
data.forEach((item: any) => {
if (activeColumnName) {
if (!item?.[activeColumnName]) {
throwError("COMMON", 400, "VALIDATION_ERROR", `Data at row ${item?.["!row#number!"]} have missing value in ${activeColumnName}`);
}
if (item?.[activeColumnName] != "Active" && item?.[activeColumnName] != "Inactive") {
{
throwError("COMMON", 400, "VALIDATION_ERROR", `Data at row ${item?.["!row#number!"]} have invalid value in ${activeColumnName}, Allowed values are Active or Inactive`);
}
}
}
const active = activeColumnName ? item[activeColumnName] : "Active";
if (active == "Active" || !item?.[uniqueIdentifierColumnName])
if (!validate(item)) {
Expand Down
Loading