Storing Data In A Single Plaintext File
Plus, more links to make you a little bit smarter today.
I Made A Fundraiser For My Upcoming Book
How I Got Into Old Games
Back when I was in middle school and high school, I spent a lot of time “forcing” myself to play older NES/SNES era games. Yet, despite how much everyone around me seemed to like them, I just couldn’t get into it. That’s until I reached college, and all of a sudden everything clicked.
On Prompt-Driven Safeguarding for Large Language Models
Prepending model inputs with safety prompts is a common practice for safeguarding large language models (LLMs) against queries with harmful intents. However, the underlying working mechanisms of safety prompts have not been unraveled yet, restricting the possibility of automatically optimizing them to improve LLM safety. In this work, we investigate how LLMs’ behavior (i.e., complying with or refusing user queries) is affected by safety prompts from the perspective of model representation. We find that in the representation space, the input queries are typically moved by safety prompts in a “higher-refusal” direction, in which models become more prone to refusing to provide assistance, even when the queries are harmless. On the other hand, LLMs are naturally capable of distinguishing harmful and harmless queries without safety prompts. Inspired by these findings, we propose a method for safety prompt optimization, namely DRO (Directed Representation Optimization). Treating a safety prompt as continuous, trainable embeddings, DRO learns to move the queries’ representations along or opposite the refusal direction, depending on their harmfulness. Experiments with eight LLMs on out-of-domain and jailbreak benchmarks demonstrate that DRO remarkably improves the safeguarding performance of human-crafted safety prompts, without compromising the models’ general performance
Large Language Models Can Continue Evolving From Mistakes
As world knowledge evolves and new task schemas emerge, Continual Learning (CL) is crucial for keeping Large Language Models (LLMs) up-to-date and addressing their shortcomings. LLMs typically require continual instruction tuning (CIT) and continual pretraining (CPT) to adapt to new tasks and acquire essential knowledge. However, collecting sufficient CPT data while addressing knowledge gaps remains challenging, as does optimizing the efficiency of utilizing this data. Inspired by the ‘summarizing mistakes’ strategy, we propose the Continue Evolving from Mistakes (CEM) method, a data-efficient approach aiming to collect CPT data and continually improve LLMs’ performance through iterative evaluation and supplementation with mistake-relevant knowledge. To enhance data utilization and mitigate forgetting, we introduce a novel training paradigm that combines CIT and CPT data. Experiments demonstrate that CEM significantly enhances model performance and continual evolution.
Storing Data In A Single Plaintext File
All tabular knowledge can be stored in a single long plain text file.
The only syntax characters needed are spaces and newlines.
This has many advantages over existing binary storage formats.
Using the method below, a very long scroll could be made containing all tabular scientific knowledge in a computable form.
Comparison of BERT vs GPT
The paper benchmarks several Transformer models [4], to show how these models can judge sentiment from a news event. This signal can then be used for downstream modelling and signal identification for commodity trading. We find that fine-tuned BERT models outperform fine-tuned or vanilla GPT models on this task. Transformer models have revolutionized the field of natural language processing (NLP) in recent years, achieving state-of-the-art results on various tasks such as machine translation, text summarization, question answering, and natural language generation. Among the most prominent transformer models are Bidirectional Encoder Representations from Transformers (BERT) and Generative Pre-trained Transformer (GPT), which differ in their architectures and objectives.
A novel portfolio construction strategy based on the core periphery profile of stocks
This paper highlights the significance of mesoscale structures, particularly the core-periphery structure, in financial networks for portfolio optimization. We build portfolios of stocks belonging to the periphery part of the Planar maximally filtered subgraphs of the underlying network of stocks created from Pearson correlations between pairs of stocks and compare its performance with some well-known strategies of Pozzi et. al. hinging around the local indices of centrality in terms of the Sharpe ratio, returns and standard deviation. Our findings reveal that these portfolios consistently outperform traditional strategies and further the core-periphery profile obtained is statistically significant across time periods. These empirical findings substantiate the efficacy of using the core-periphery profile of the stock market network for both inter-day and intraday trading and provide valuable insights for investors seeking better returns.