feat(description): don't truncate by default

packages/metascraper-description/README.md

@@ -23,7 +23,7 @@ $ npm install metascraper-description --save
 ##### truncateLength
 
 Type: `number`<br>
-Default: `300`
+Default: `Number.MAX_SAFE_INTEGER`
 
 Truncate the value extracted to a maximum size.
 

packages/metascraper-helpers/index.js

@@ -115,8 +115,6 @@ const REGEX_LOCATION = /^[A-Z\s]+\s+[-—–]\s+/
 
 const REGEX_TITLE_SEPARATOR = /^[^|\-/•—]+/
 
-const TRUNCATE_MAX_LENGTH = 300
-
 const AUTHOR_MAX_LENGTH = 128
 
 const removeLocation = value => replace(value, REGEX_LOCATION, '')
@@ -253,7 +251,7 @@ const description = (value, opts) =>
 
 const getDescription = (
   str,
-  { truncateLength = TRUNCATE_MAX_LENGTH, ellipsis = '…', ...opts } = {}
+  { truncateLength = Number.MAX_SAFE_INTEGER, ellipsis = '…', ...opts } = {}
 ) => {
   let truncated = str.slice(0, truncateLength)
   if (truncated.length < str.length) truncated = truncated.trim() + ellipsis

packages/metascraper-helpers/test/index.js

@@ -58,11 +58,32 @@ test('.normalizeUrl', t => {
   )
   t.is(normalizeUrl('https://example.com/'), 'https://example.com/')
   t.is(normalizeUrl('https://example.com'), 'https://example.com/')
-  t.is(normalizeUrl('https://www.example.com', 'https://www.example.com/foo'), 'https://www.example.com/foo')
-  t.is(normalizeUrl('https://www.example.com', '/foo'), 'https://www.example.com/foo')
-  t.is(normalizeUrl('https://www.example.com', 'file.html'), 'https://www.example.com/file.html')
-  t.is(normalizeUrl('https://www.example.com', 'data:text/html;base64,PGh0bWw+SGVsbG8sIHdvcmxkITwvaHRtbD4='), 'data:text/html;base64,PGh0bWw+SGVsbG8sIHdvcmxkITwvaHRtbD4=')
-  t.is(normalizeUrl('https://www.example.com', 'javascript:alert(\'Hello, world!\');'), undefined)
+  t.is(
+    normalizeUrl('https://www.example.com', 'https://www.example.com/foo'),
+    'https://www.example.com/foo'
+  )
+  t.is(
+    normalizeUrl('https://www.example.com', '/foo'),
+    'https://www.example.com/foo'
+  )
+  t.is(
+    normalizeUrl('https://www.example.com', 'file.html'),
+    'https://www.example.com/file.html'
+  )
+  t.is(
+    normalizeUrl(
+      'https://www.example.com',
+      'data:text/html;base64,PGh0bWw+SGVsbG8sIHdvcmxkITwvaHRtbD4='
+    ),
+    'data:text/html;base64,PGh0bWw+SGVsbG8sIHdvcmxkITwvaHRtbD4='
+  )
+  t.is(
+    normalizeUrl(
+      'https://www.example.com',
+      "javascript:alert('Hello, world!');"
+    ),
+    undefined
+  )
   t.is(normalizeUrl('https://www.example.com', 'javascript:void(0)'), undefined)
 })
 
@@ -277,10 +298,12 @@ test('.isImageExtension', t => {
 test('.description', t => {
   t.is(
     description(
-      'Let me tell you the story of two investors, neither of whom knew each other, but whose paths crossed in an interesting way. Grace Groner was orphaned at age 12. She never married. She never had kids. She never drove a car. She lived most of her life alone in a one-bedroom house and worked her whole career as a secretary. She was, by all accounts, a lovely lady. But she lived a humble and quiet life. That made the $7 million she left to charity after her death in 2010 at age 100 all the more confusing. People who knew her asked: Where did Grace get all that money? But there was no secret. There was no inheritance. Grace took humble savings from a meager salary and enjoyed eighty years of hands-off compounding in the stock market. That was it. Weeks after Grace died, an unrelated investing story hit the news. Richard Fuscone, former vice chairman of Merrill Lynch’s Latin America division, declared personal bankruptcy, fighting off foreclosure on two homes, one of which was nearly 20,000 square feet and had a $66,000 a month mortgage. Fuscone was the opposite of Grace Groner; educated at Harvard and University of Chicago, he became so successful in the investment industry that he retired in his 40s to “pursue personal and charitable interests.” But heavy borrowing and illiquid investments did him in. The same year Grace Goner left a veritable fortune to charity, Richard stood before a bankruptcy judge and declared: “I have been devastated by the financial crisis … The only source of liquidity is whatever my wife is able to sell in terms of personal furnishings.” The purpose of these stories is not to say you should be like Grace and avoid being like Richard. It’s to point out that there is no other field where these stories are even possible. In what other field does someone with no education, no relevant experience, no resources, and no connections vastly outperform someone with the best education, the most relevant experiences, the best resources and the best connections? There will never be a story of a Grace Groner performing heart surgery better than a Harvard-trained cardiologist. Or building a faster chip than Apple’s engineers. Unthinkable. But these stories happen in investing. That’s because investing is not the study of finance. It’s the study of how people behave with money. And behavior is hard to teach, even to really smart people. You can’t sum up behavior with formulas to memorize or spreadsheet models to follow. Behavior is inborn, varies by person, is hard to measure, changes over time, and people are prone to deny its existence, especially when describing themselves. Grace and Richard show that managing money isn’t necessarily about what you know; it’s how you behave. But that’s not how finance is typically taught or discussed. The finance industry talks too much about what to do, and not enough about what happens in your head when you try to do it. This report describes 20 flaws, biases, and causes of bad behavior I’ve seen pop up often when people deal with money. 1. Earned success and deserved failure fallacy: A tendency to underestimate the role of luck and risk, and a failure to recognize that luck and risk are different sides of the same coin. I like to ask people, “What do you want to know about investing that we can’t know?” It’s not a practical question. So few people ask it. But it forces anyone you ask to think about what they intuitively think is true but don’t spend much time trying to answer because it’s futile. Years ago I asked economist Robert Shiller the question. He answered, “The exact role of luck in successful outcomes.” I love that, because no one thinks luck doesn’t play a role in financial success. But since it’s hard to quantify luck, and rude to suggest people’s success is owed to luck, the default stance is often to implicitly ignore luck as a factor. If I say, “There are a billion investors in the world. By sheer chance, would you expect 100 of them to become billionaires predominately off luck?” You would reply, “Of course.” But then if I ask you to name those investors – to their face – you will back down. That’s the problem. The same goes for failure. Did failed businesses not try hard enough? Were bad investments not thought through well enough?'
+      'Let me tell you the story of two investors, neither of whom knew each other, but whose paths crossed in an interesting way. Grace Groner was orphaned at age 12. She never married. She never had kids. She never drove a car. She lived most of her life alone in a one-bedroom house and worked her whole career as a secretary. She was, by all accounts, a lovely lady. But she lived a humble and quiet life. That made the $7 million she left to charity after her death in 2010 at age 100 all the more confusing. People who knew her asked: Where did Grace get all that money? But there was no secret. There was no inheritance. Grace took humble savings from a meager salary and enjoyed eighty years of hands-off compounding in the stock market. That was it. Weeks after Grace died, an unrelated investing story hit the news. Richard Fuscone, former vice chairman of Merrill Lynch’s Latin America division, declared personal bankruptcy, fighting off foreclosure on two homes, one of which was nearly 20,000 square feet and had a $66,000 a month mortgage. Fuscone was the opposite of Grace Groner; educated at Harvard and University of Chicago, he became so successful in the investment industry that he retired in his 40s to “pursue personal and charitable interests.” But heavy borrowing and illiquid investments did him in. The same year Grace Goner left a veritable fortune to charity, Richard stood before a bankruptcy judge and declared: “I have been devastated by the financial crisis … The only source of liquidity is whatever my wife is able to sell in terms of personal furnishings.” The purpose of these stories is not to say you should be like Grace and avoid being like Richard. It’s to point out that there is no other field where these stories are even possible. In what other field does someone with no education, no relevant experience, no resources, and no connections vastly outperform someone with the best education, the most relevant experiences, the best resources and the best connections? There will never be a story of a Grace Groner performing heart surgery better than a Harvard-trained cardiologist. Or building a faster chip than Apple’s engineers. Unthinkable. But these stories happen in investing. That’s because investing is not the study of finance. It’s the study of how people behave with money. And behavior is hard to teach, even to really smart people. You can’t sum up behavior with formulas to memorize or spreadsheet models to follow. Behavior is inborn, varies by person, is hard to measure, changes over time, and people are prone to deny its existence, especially when describing themselves. Grace and Richard show that managing money isn’t necessarily about what you know; it’s how you behave. But that’s not how finance is typically taught or discussed. The finance industry talks too much about what to do, and not enough about what happens in your head when you try to do it. This report describes 20 flaws, biases, and causes of bad behavior I’ve seen pop up often when people deal with money. 1. Earned success and deserved failure fallacy: A tendency to underestimate the role of luck and risk, and a failure to recognize that luck and risk are different sides of the same coin. I like to ask people, “What do you want to know about investing that we can’t know?” It’s not a practical question. So few people ask it. But it forces anyone you ask to think about what they intuitively think is true but don’t spend much time trying to answer because it’s futile. Years ago I asked economist Robert Shiller the question. He answered, “The exact role of luck in successful outcomes.” I love that, because no one thinks luck doesn’t play a role in financial success. But since it’s hard to quantify luck, and rude to suggest people’s success is owed to luck, the default stance is often to implicitly ignore luck as a factor. If I say, “There are a billion investors in the world. By sheer chance, would you expect 100 of them to become billionaires predominately off luck?” You would reply, “Of course.” But then if I ask you to name those investors – to their face – you will back down. That’s the problem. The same goes for failure. Did failed businesses not try hard enough? Were bad investments not thought through well enough?',
+      { truncateLength: 300 }
     ),
     'Let me tell you the story of two investors, neither of whom knew each other, but whose paths crossed in an interesting way. Grace Groner was orphaned at age 12. She never married. She never had kids. She never drove a car. She lived most of her life alone in a one-bedroom house and worked her whole…'
   )
+
   t.is(
     description(
       'He talks about the ocean and his favorite animal the cowrie...'