Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bug: The data type was not determined during load data #852

Closed
2 of 3 tasks
shangyanwen opened this issue Oct 31, 2022 · 5 comments · Fixed by #1105
Closed
2 of 3 tasks

bug: The data type was not determined during load data #852

shangyanwen opened this issue Oct 31, 2022 · 5 comments · Fixed by #1105
Assignees
@lujiashun
Labels
A-bug Something isn't working prio: low Low priority
Milestone
StoneDB_5.7_v1.0.2

Comments

@shangyanwen
Copy link
Contributor

Have you read the Contributing Guidelines on issues?

Please confirm if bug report does NOT exists already ?

  • I confirm there is no existing issue for this

Describe the problem

The type of the field in column A is int, and the data is A string. No error is reported when the string data is imported into column A, and the import is successful

Expected behavior

Verify data types. When data types are inconsistent, an error message is displayed

#innodb error message：ERROR 1366 (HY000): Incorrect integer value: 'ECONOMY POLISHED STEEL' for column 'P_SIZE' at row 1

How To Reproduce

##create table
CREATE TABLE IF NOT EXISTS `lineorder_flat` (
  `LO_ORDERDATE` date NOT NULL COMMENT '',
  `LO_ORDERKEY` int(11) NOT NULL COMMENT '',
  `LO_LINENUMBER` tinyint(4) NOT NULL COMMENT '',
   `LO_CUSTKEY` int(11) NOT NULL COMMENT '',
  `LO_PARTKEY` int(11) NOT NULL COMMENT '',
  `LO_SUPPKEY` int(11) NOT NULL COMMENT '',
  `LO_ORDERPRIORITY` varchar(100) NOT NULL COMMENT '',
  `LO_SHIPPRIORITY` tinyint(4) NOT NULL COMMENT '',
  `LO_QUANTITY` tinyint(4) NOT NULL COMMENT '',
  `LO_EXTENDEDPRICE` int(11) NOT NULL COMMENT '',
  `LO_ORDTOTALPRICE` int(11) NOT NULL COMMENT '',
  `LO_DISCOUNT` tinyint(4) NOT NULL COMMENT '',
  `LO_REVENUE` int(11) NOT NULL COMMENT '',
  `LO_SUPPLYCOST` int(11) NOT NULL COMMENT '',
  `LO_TAX` tinyint(4) NOT NULL COMMENT '',
  `LO_COMMITDATE` date NOT NULL COMMENT '',
  `LO_SHIPMODE` varchar(100) NOT NULL COMMENT '',
  `C_NAME` varchar(100) NOT NULL COMMENT '',
  `C_ADDRESS` varchar(100) NOT NULL COMMENT '',
  `C_CITY` varchar(100) NOT NULL COMMENT '',
  `C_NATION` varchar(100) NOT NULL COMMENT '',
  `C_REGION` varchar(100) NOT NULL COMMENT '',
  `C_PHONE` varchar(100) NOT NULL COMMENT '',
  `C_MKTSEGMENT` varchar(100) NOT NULL COMMENT '',
  `S_NAME` varchar(100) NOT NULL COMMENT '',
  `S_ADDRESS` varchar(100) NOT NULL COMMENT '',
  `S_CITY` varchar(100) NOT NULL COMMENT '',
  `S_NATION` varchar(100) NOT NULL COMMENT '',
  `S_REGION` varchar(100) NOT NULL COMMENT '',
  `S_PHONE` varchar(100) NOT NULL COMMENT '',
  `P_NAME` varchar(100) NOT NULL COMMENT '',
  `P_MFGR` varchar(100) NOT NULL COMMENT '',
  `P_CATEGORY` varchar(100) NOT NULL COMMENT '',
  `P_BRAND` varchar(100) NOT NULL COMMENT '',
  `P_COLOR` varchar(100) NOT NULL COMMENT '',
  `P_TYPE` varchar(100) NOT NULL COMMENT '',
  `P_SIZE` tinyint(4) NOT NULL COMMENT '',
  `P_CONTAINER` varchar(100) NOT NULL COMMENT ''
);

##load data

19980721,34,1,12202,88362,727,3-MEDIUM,0,13,1755468,5894967,0,1755468,81021,7,19980914,REG AIR,Customer#000012202,crWpqvyx4fOpC3IP,JAPAN    5,JAPAN,ASIA,22-437-209-5615,FURNITURE,Supplier#000000727,HO,lDAVvCDTthJs3oYHW,BRAZIL   4,BRAZIL,AMERICA,12-815-316-3525,dim thistle,MFGR#2,MFGR#23,MFGR#236,seashell,ECONOMY POLISHED STEEL,39,WRAP BAG

Environment

Ver 14.14 Distrib 5.7.36-StoneDB, for Linux (x86_64) using EditLine wrapper

Are you interested in submitting a PR to solve the problem?

  • Yes, I will!
@shangyanwen shangyanwen added A-bug Something isn't working prio: low Low priority labels Oct 31, 2022
@shangyanwen shangyanwen added this to the stonedb_5.7_v1.0.2 milestone Oct 31, 2022
@RingsC RingsC assigned lujiashun and unassigned duanfuxiang0 Nov 29, 2022
@RingsC
Copy link
Contributor

RingsC commented Dec 1, 2022

refs to issue :#656 , when we are loading data from outfile, and the data will overflow, a warning will be issued.

@lujiashun
Copy link

ACK

@lujiashun
Copy link

reference: #1000

@lujiashun
Copy link

lujiashun commented Dec 6, 2022 •
edited
Loading

enclose related issue, related issue with :#1006

@lujiashun
Copy link 

`LO_ORDERDATE` date NOT NULL COMMENT '',                 <------>    19980721,
`LO_ORDERKEY` int(11) NOT NULL COMMENT '',               <------>    34,
`LO_LINENUMBER` tinyint(4) NOT NULL COMMENT '',          <------>    1,
`LO_CUSTKEY` int(11) NOT NULL COMMENT '',                <------>    12202,
`LO_PARTKEY` int(11) NOT NULL COMMENT '',                <------>    88362,
`LO_SUPPKEY` int(11) NOT NULL COMMENT '',                <------>    727,
`LO_ORDERPRIORITY` varchar(100) NOT NULL COMMENT '',     <------>    3-MEDIUM,
`LO_SHIPPRIORITY` tinyint(4) NOT NULL COMMENT '',        <------>    0,
`LO_QUANTITY` tinyint(4) NOT NULL COMMENT '',            <------>    13,
`LO_EXTENDEDPRICE` int(11) NOT NULL COMMENT '',          <------>    1755468,
`LO_ORDTOTALPRICE` int(11) NOT NULL COMMENT '',          <------>    5894967,
`LO_DISCOUNT` tinyint(4) NOT NULL COMMENT '',            <------>    0,
`LO_REVENUE` int(11) NOT NULL COMMENT '',                <------>    1755468,
`LO_SUPPLYCOST` int(11) NOT NULL COMMENT '',             <------>    81021,
`LO_TAX` tinyint(4) NOT NULL COMMENT '',                 <------>    7,
`LO_COMMITDATE` date NOT NULL COMMENT '',                <------>    19980914,
`LO_SHIPMODE` varchar(100) NOT NULL COMMENT '',          <------>    REG AIR,
`C_NAME` varchar(100) NOT NULL COMMENT '',               <------>    Customer#000012202,
`C_ADDRESS` varchar(100) NOT NULL COMMENT '',            <------>    crWpqvyx4fOpC3IP,
`C_CITY` varchar(100) NOT NULL COMMENT '',               <------>    JAPAN    5,
`C_NATION` varchar(100) NOT NULL COMMENT '',             <------>    JAPAN,
`C_REGION` varchar(100) NOT NULL COMMENT '',             <------>    ASIA,
`C_PHONE` varchar(100) NOT NULL COMMENT '',              <------>    22-437-209-5615,
`C_MKTSEGMENT` varchar(100) NOT NULL COMMENT '',         <------>    FURNITURE,
`S_NAME` varchar(100) NOT NULL COMMENT '',               <------>    Supplier#000000727,
`S_ADDRESS` varchar(100) NOT NULL COMMENT '',            <------>    HO,lDAVvCDTthJs3oYHW,
`S_CITY` varchar(100) NOT NULL COMMENT '',               <------>    BRAZIL   4,
`S_NATION` varchar(100) NOT NULL COMMENT '',             <------>    BRAZIL,
`S_REGION` varchar(100) NOT NULL COMMENT '',             <------>    AMERICA,
`S_PHONE` varchar(100) NOT NULL COMMENT '',              <------>    12-815-316-3525,
`P_NAME` varchar(100) NOT NULL COMMENT '',               <------>    dim thistle,
`P_MFGR` varchar(100) NOT NULL COMMENT '',               <------>    MFGR#2,
`P_CATEGORY` varchar(100) NOT NULL COMMENT '',           <------>    MFGR#23,
`P_BRAND` varchar(100) NOT NULL COMMENT '',              <------>    MFGR#236,
`P_COLOR` varchar(100) NOT NULL COMMENT '',              <------>    seashell,
`P_TYPE` varchar(100) NOT NULL COMMENT '',               <------>    ECONOMY POLISHED STEEL,
`P_SIZE` tinyint(4) NOT NULL COMMENT '',                 <------>    39,
`P_CONTAINER` varchar(100) NOT NULL COMMENT ''           <------>    WRAP BAG

lujiashun pushed a commit to lujiashun/stonedb that referenced this issue Dec 9, 2022
feat(tianmu): fix bug: The data type was not determined during load d…
99f6011 
…ata. (stoneatom#852)

[summary]
function GetValue should not truncate string from delimiter character;
e.g,  string "HO,lDAVvCDTthJs3oYHW" enclosed by double-quote, be should stored as a whole;
@lujiashun lujiashun mentioned this issue Dec 9, 2022
Merged
14 tasks
lujiashun pushed a commit to lujiashun/stonedb that referenced this issue Dec 12, 2022
feat(tianmu): fix bug: The data type was not determined during load d…
54b3bee 
…ata. (stoneatom#852)

[summary]
function GetValue should not truncate string from delimiter character;
e.g,  string "HO,lDAVvCDTthJs3oYHW" enclosed by double-quote, be should stored as a whole;
@mergify mergify bot closed this as completed in #1105 Dec 13, 2022
mergify bot pushed a commit that referenced this issue Dec 13, 2022
@mergify
feat(tianmu): fix bug: The data type was not determined during load d…
3398895 
…ata. (#852)

[summary]
function GetValue should not truncate string from delimiter character;
e.g,  string "HO,lDAVvCDTthJs3oYHW" enclosed by double-quote, be should stored as a whole;
@mergify mergify bot moved this to Done in StoneDB for MySQL 5.7 Dec 13, 2022
konghaiya pushed a commit to konghaiya/stonedb that referenced this issue Mar 7, 2023
@konghaiya
feat(tianmu): fix bug: The data type was not determined during load d…
b3fab95 
…ata. (stoneatom#852)

[summary]
function GetValue should not truncate string from delimiter character;
e.g,  string "HO,lDAVvCDTthJs3oYHW" enclosed by double-quote, be should stored as a whole;
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@lujiashun lujiashun

Labels
A-bug Something isn't working prio: low Low priority
Projects
StoneDB for MySQL 5.7
Status: Done
Milestone
StoneDB_5.7_v1.0.2
Development

Successfully merging a pull request may close this issue.

feat(tianmu): fix bug: The data type was not determined during load data. (#852) lujiashun/stonedb
4 participants
@RingsC @duanfuxiang0 @shangyanwen @lujiashun