Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Header validation #861

Merged
merged 32 commits into from
Jun 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
ea9731f
change in filter recursive
nitish-egov May 21, 2024
f761d21
Merge branch 'campaign' of https://github.com/egovernments/DIGIT-Fron…
nitish-egov May 21, 2024
cefa966
lowest level
nitish-egov May 21, 2024
577a6a0
Merge branch 'campaign' of https://github.com/egovernments/DIGIT-Fron…
nitish-egov May 21, 2024
99e8460
Merge branch 'campaign' of https://github.com/egovernments/DIGIT-Fron…
nitish-egov May 22, 2024
40e11f5
Merge branch 'campaign' of https://github.com/egovernments/DIGIT-Fron…
nitish-egov May 23, 2024
5a0b0cd
Merge branch 'campaign' of https://github.com/egovernments/DIGIT-Fron…
nitish-egov May 28, 2024
b9be47e
Merge branch 'campaign' of https://github.com/egovernments/DIGIT-Fron…
nitish-egov May 28, 2024
ca07874
Merge branch 'campaign' of https://github.com/egovernments/DIGIT-Fron…
nitish-egov May 28, 2024
0f9d7d5
Merge branch 'campaign' of https://github.com/egovernments/DIGIT-Fron…
nitish-egov May 29, 2024
5207560
Merge branch 'campaign' of https://github.com/egovernments/DIGIT-Fron…
nitish-egov May 29, 2024
63249b9
Merge branch 'campaign' of https://github.com/egovernments/DIGIT-Fron…
nitish-egov May 29, 2024
62aff8e
Merge branch 'campaign' of https://github.com/egovernments/DIGIT-Fron…
nitish-egov May 29, 2024
3903614
Merge branch 'campaign' of https://github.com/egovernments/DIGIT-Fron…
nitish-egov May 30, 2024
87cd5ee
Merge branch 'campaign' of https://github.com/egovernments/DIGIT-Fron…
nitish-egov May 30, 2024
b181c14
Merge branch 'campaign' of https://github.com/egovernments/DIGIT-Fron…
nitish-egov May 30, 2024
4df4263
Merge branch 'campaign' of https://github.com/egovernments/DIGIT-Fron…
nitish-egov May 31, 2024
d7ab75d
Merge branch 'campaign' of https://github.com/egovernments/DIGIT-Fron…
nitish-egov Jun 3, 2024
3e908cc
Merge branch 'campaign' of https://github.com/egovernments/DIGIT-Fron…
nitish-egov Jun 5, 2024
0009fb7
Merge branch 'campaign' of https://github.com/egovernments/DIGIT-Fron…
nitish-egov Jun 7, 2024
cf9a8f4
Merge branch 'campaign' of https://github.com/egovernments/DIGIT-Fron…
nitish-egov Jun 7, 2024
80cc2da
Merge branch 'campaign' of https://github.com/egovernments/DIGIT-Fron…
nitish-egov Jun 10, 2024
a7cad1c
Merge branch 'campaign' of https://github.com/egovernments/DIGIT-Fron…
nitish-egov Jun 11, 2024
0b86778
Merge branch 'campaign' of https://github.com/egovernments/DIGIT-Fron…
nitish-egov Jun 11, 2024
af9e3aa
Merge branch 'campaign' of https://github.com/egovernments/DIGIT-Fron…
nitish-egov Jun 11, 2024
019445b
Merge branch 'campaign' of https://github.com/egovernments/DIGIT-Fron…
nitish-egov Jun 11, 2024
e62cd7b
Merge remote-tracking branch 'origin/campaign' into campaign
nitish-egov Jun 12, 2024
09d56b8
Merge branch 'campaign' of https://github.com/egovernments/DIGIT-Fron…
nitish-egov Jun 12, 2024
e89f25e
removed unnecessary validation for target
nitish-egov Jun 12, 2024
a746a0d
Merge branch 'campaign' of https://github.com/egovernments/DIGIT-Fron…
nitish-egov Jun 13, 2024
4b08d74
changed the logic of header validation
nitish-egov Jun 13, 2024
12f85ec
space refactor
nitish-egov Jun 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions utilities/project-factory/src/server/api/campaignApis.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import { httpRequest } from "../utils/request";
import { getFormattedStringForDebug, logger } from "../utils/logger";
import createAndSearch from '../config/createAndSearch';
import { getDataFromSheet, generateActivityMessage, throwError, translateSchema, replicateRequest } from "../utils/genericUtils";
import { immediateValidationForTargetSheet, validateSheetData, validateTargetSheetData } from '../validators/campaignValidators';
import { immediateValidationForTargetSheet, validateSheetData, validateTargetSheetData } from '../validators/campaignValidators';
import { callMdmsTypeSchema, getCampaignNumber } from "./genericApis";
import { boundaryBulkUpload, convertToTypeData, generateHierarchy, generateProcessedFileAndPersist, getLocalizedName, reorderBoundariesOfDataAndValidate } from "../utils/campaignUtils";
const _ = require('lodash');
Expand Down Expand Up @@ -497,8 +497,6 @@ async function processValidate(request: any, localizationMap?: { [key: string]:
immediateValidationForTargetSheet(dataFromSheet, localizationMap);
logger.info("target sheet format validation completed and starts with data validation");
validateTargetSheetData(dataFromSheet, request, createAndSearchConfig?.boundaryValidation, localizationMap);
logger.info("target sheet data validation completed");

}

else {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -411,6 +411,7 @@ async function validateSheetData(data: any, request: any, schema: any, boundaryV
async function validateTargetSheetData(data: any, request: any, boundaryValidation: any, localizationMap?: any) {
try {
const errors: any[] = [];
await validateHeadersOfTargetSheet(request, localizationMap);
if (boundaryValidation) {
// const localizedBoundaryValidationColumn = getLocalizedName(boundaryValidation?.column, localizationMap)
// await validateTargetBoundaryData(data, request, localizedBoundaryValidationColumn, errors, localizationMap);
Expand All @@ -421,13 +422,27 @@ async function validateTargetSheetData(data: any, request: any, boundaryValidati
request.body.ResourceDetails.status = resourceDataStatuses.invalid;
}
await generateProcessedFileAndPersist(request, localizationMap);
logger.info("target sheet data validation completed");
}
catch (error) {
console.log(error)
await handleResouceDetailsError(request, error);
}
}


async function validateHeadersOfTargetSheet(request: any, localizationMap?: any) {
const fileUrl = await validateFile(request);
const targetWorkbook: any = await getTargetWorkbook(fileUrl);
const hierarchy = await getHierarchy(request, request?.body?.ResourceDetails?.tenantId, request?.body?.ResourceDetails?.hierarchyType);
const finalValidHeadersForTargetSheetAsPerCampaignType = await getFinalValidHeadersForTargetSheetAsPerCampaignType(request, hierarchy, localizationMap);
logger.info("finalValidHeadersForTargetSheetAsPerCampaignType :" + JSON.stringify(finalValidHeadersForTargetSheetAsPerCampaignType));
logger.info("validating headers of target sheet started")
validateHeadersOfTabsWithTargetInTargetSheet(targetWorkbook, finalValidHeadersForTargetSheetAsPerCampaignType);
logger.info("validation of target sheet headers completed")
}


function validateBooleanField(obj: any, fieldName: any, index: any) {
if (!obj.hasOwnProperty(fieldName)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace hasOwnProperty with Object.hasOwn() to avoid potential issues with objects that do not have hasOwnProperty as a direct property.

- if (obj.hasOwnProperty('key')) {
+ if (Object.hasOwn(obj, 'key')) {

Also applies to: 457-457, 472-472

Tools
Biome

[error] 447-447: Do not access Object.prototype method 'hasOwnProperty' from target object. (lint/suspicious/noPrototypeBuiltins)

It's recommended using Object.hasOwn() instead of using Object.hasOwnProperty().
See MDN web docs for more details.

throwError("COMMON", 400, "VALIDATION_ERROR", `Object at index ${index} is missing field "${fieldName}".`);
Expand Down Expand Up @@ -516,17 +531,17 @@ async function validateCreateRequest(request: any, localizationMap?: any) {
if (request.body.ResourceDetails.type == 'boundary') {
await validateBoundarySheetData(request, fileUrl, localizationMap);
}
if (request?.body?.ResourceDetails?.type == 'boundaryWithTarget') {
const targetWorkbook: any = await getTargetWorkbook(fileUrl);
const hierarchy = await getHierarchy(request, request?.body?.ResourceDetails?.tenantId, request?.body?.ResourceDetails?.hierarchyType);
const finalValidHeadersForTargetSheetAsPerCampaignType = await getFinalValidHeadersForTargetSheetAsPerCampaignType(request, hierarchy, localizationMap);
logger.info("finalValidHeadersForTargetSheetAsPerCampaignType :" + JSON.stringify(finalValidHeadersForTargetSheetAsPerCampaignType));
validateTabsWithTargetInTargetSheet(targetWorkbook, finalValidHeadersForTargetSheetAsPerCampaignType);
}
// if (request?.body?.ResourceDetails?.type == 'boundaryWithTarget') {
// const targetWorkbook: any = await getTargetWorkbook(fileUrl);
// const hierarchy = await getHierarchy(request, request?.body?.ResourceDetails?.tenantId, request?.body?.ResourceDetails?.hierarchyType);
// const finalValidHeadersForTargetSheetAsPerCampaignType = await getFinalValidHeadersForTargetSheetAsPerCampaignType(request, hierarchy, localizationMap);
// logger.info("finalValidHeadersForTargetSheetAsPerCampaignType :" + JSON.stringify(finalValidHeadersForTargetSheetAsPerCampaignType));
// validateTabsWithTargetInTargetSheet(targetWorkbook, finalValidHeadersForTargetSheetAsPerCampaignType);
// }
}
}

function validateTabsWithTargetInTargetSheet(targetWorkbook: any, expectedHeadersForTargetSheet: any) {
function validateHeadersOfTabsWithTargetInTargetSheet(targetWorkbook: any, expectedHeadersForTargetSheet: any) {
targetWorkbook.eachSheet((worksheet: any, sheetId: any) => {
if (sheetId > 2) { // Starting from the second sheet
// Convert the sheet to an array of headers
Expand Down Expand Up @@ -1134,8 +1149,10 @@ async function validateDownloadRequest(request: any) {
await validateHierarchyType(request, hierarchyType, tenantId);
}

function immediateValidationForTargetSheet(dataFromSheet: any, localizationMap: any) {
async function immediateValidationForTargetSheet(dataFromSheet: any, localizationMap: any) {
logger.info("validating all district tabs present started")
validateAllDistrictTabsPresentOrNot(dataFromSheet, localizationMap);
logger.info("validation of all district tabs present completed")
for (const key in dataFromSheet) {
if (key !== getLocalizedName(getBoundaryTabName(), localizationMap) && key !== getLocalizedName(config?.values?.readMeTab, localizationMap)) {
if (Object.prototype.hasOwnProperty.call(dataFromSheet, key)) {
Expand Down
Loading