notes updates

_includes/01_research.html

@@ -153,6 +153,18 @@ <h2 style="text-align: center; margin-top: -150px;"> Research</h2>
         </tr>
     </thead>
     <tbody>
+        <tr>
+            <td class="center">'24</td>
+            <td>Interpretable Language Modeling via Induction-head Ngram Models
+            </td>
+            <td>kim*, mantena*, et al.</td>
+            <td class="med">🧠🔎🌀</td>
+            <td class="center"><a href="https://arxiv.org/abs/2411.00066">arxiv</a></td>
+            <td class="big"><a href="https://github.com/ejkim47/induction-gram"><i class="fa fa-github fa-fw"></i></a>
+            </td>
+            <td class="med">
+            </td>
+        </tr>
         <tr>
             <td class="center">'24</td>
             <td>A generative framework to bridge data-driven models and scientific theories in language neuroscience
@@ -254,7 +266,7 @@ <h2 style="text-align: center; margin-top: -150px;"> Research</h2>
             </td>
             <td>gero et al.</td>
             <td class="med">🔎🌀</td>
-            <td class="center"><a href="https://arxiv.org/abs/2403.01002">arxiv</a></td>
+            <td class="center"><a href="https://arxiv.org/abs/2403.01002">ml4h findings</a></td>
             <td class="big"><a href="https://github.com/microsoft/attribute-structuring/"><i
                         class="fa fa-github fa-fw"></i></a></td>
             <td class="med">

_notes/ai/psychology.md

@@ -669,7 +669,18 @@ subtitle: Some notes on papers / books surrounding psychology, especially evolut
 - privacy (spaces in people’s psyches that everyone needs in healthy relationships) and secrecy (which stems from shame and tends to be corrosive)
 - In the best goodbyes, there’s always the feeling that there’s something more to say
 
-
+# the gifted child (alice miller)
+
+- "A child cannot run away from her as her own mother once did. A child can be so brought up that it becomes what she wants it to be."
+- "a child can never see through unconscious manipulation. It is like the air he breathes; he knows no other, and it appears to him to be the only normal possiblity"
+- *cathexis* - the concentration of mental energy on onen particular person, idea, or object (esp. to an unhealthy degree)
+- *grandiosity* - the person who is "grandiose" is admired everywhere and needs this admiration; indeed, he cannot live without it.
+  - "And is he noto bound to carry pent-up rage within himself, against those who have made him afraid to walk without stilts?"
+- *introjection* - the unconscious adoption of the ideas or attitudes of others
+- sisyphean - (of a task) such that it can never be completed.
+- depression ~ a possible reaction to psychic pain caused by the discrepancy between the actual and the ideal self representation
+- transference - the redicirection to a substitute, usually a therapist, of emotions that were originally felt in childhood
+  - countertransference - the emotional reaction of the analyst to the subject's contribution
 
 # attached (amir levine & rachel heller)
 

_notes/research_ovws/ovw_llms.md

@@ -535,6 +535,7 @@ Model merging (some of these are non-transformer papers) = combine different mod
     - improves perplexities, when controlling for training cost
       - require expert domain specialization
     - Cluster-Branch-Train-Merge ([gururangan...smith, zettlemoyer, 2023](https://arxiv.org/abs/2303.14177)) - start by clustering data to do unsupervised domain discovery
+  - LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Merging ([wang...frossard, 2024](https://arxiv.org/abs/2410.17146)) - updating deeper layers more than shallow layers helps prevent forgetting across tasks
 - fit many models into one
   - superposition of many models into one ([cheung...olshausen, 2019](https://proceedings.neurips.cc/paper/2019/hash/4c7a167bb329bd92580a99ce422d6fa6-Abstract.html)) - both during training/testing models are indexed via a high-dim key for each task
   - supermasks in superposition ([wortsman, ..., yosinski, farhadi, 2020](https://proceedings.neurips.cc/paper/2020/hash/ad1f8bb9b51f023cdc80cf94bb615aa9-Abstract.html)) - randomly fixed base net + for each task finds subnet that performs well
@@ -716,6 +717,10 @@ Editing is generally very similar to just adaptation/finetuning. One distinction
 - [transformer-debugger](https://github.com/openai/transformer-debugger) (openAI)
   - neuronpedia: visualization tool for neuron SAEs ([lin & bloom, 2024](https://www.lesswrong.com/posts/BaEQoxHhWPrkinmxd/announcing-neuronpedia-as-a-platform-to-accelerate-research))
   - Improving Dictionary Learning with Gated Sparse Autoencoders ([rajamanoharan...nanda, 2024](https://arxiv.org/pdf/2404.16014))
+  - Automatically Interpreting Millions of Features in Large Language Models ([paulo...belrose, 2024](https://arxiv.org/abs/2410.13928))
+- Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking ([prakash...belinkov, bau, 2024](https://arxiv.org/abs/2402.14811)) - finetuning does not seem to change the behavior of circuits, rather just enhances them
+  - Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks ([jain...krueger, 2024](https://arxiv.org/abs/2311.12786)) - finetuning learns a fairly simple wrapper that can be reversed easily
+
 
 
 ## debugging / interpretation