US Library of Congress Recommended Storage Format

The US Library of Congress recommends SQLite for the preservation of data — not other relational database systems. I am not surprised since SQLite reads and writes data to files without the need of a server. SQLite is low maintenance and has not changed the mechanism used to read and write data since released on 08/17/2000 following SQL-92 standards. The latter means that data written quarter of a century ago can still be read today without errors and it would most likely be read without errors decades from now.

Read more.

  1. https://sqlite.org/locrsf.html
  2. https://www.loc.gov/preservation/digital/formats/fdd/fdd000461.shtml#local
  3. https://www.loc.gov/preservation/resources/rfs/data.html
  4. https://news.ycombinator.com/item?id=48042434

SQL and its relation to AI

Artificial intelligence (AI) is a series of co-related queries to retrieve data from a series of databases – often times, guessing what the end user needs.

Structure Query Language (SQL) is the most basic form of retrieving data and managing the databases that AI needs.

Although AI could possibly write the query you need, there is a chance that you would need to edit it and/or improve it. After all, some AI models anticipate what the user asks verbally and try to answer right away. The latter is annoying since these AI models use data compiled by user group data – collected from people from a certain age range, geographical location and/or other factor. As such, when a person asks for information that is not in the expected dataset (for example, a question about music sung in Russian when the model expects everyone to listen to music sung in English, worse yet without the compensation for bad pronunciation), AI returns pathetic answers. AI fails to guess. This is where you (as the developer) come in and improve the queries that AI might generate for you.

Many vendors offer their own flavors of AI. For example, MySQL has its own libraries and facilities for AI – Automated Machine Language (AutoML) and Generative AI (GenAI).

MySQL AI provides integrated, automated, and secure machine learning (ML) and generative AI capabilities. AutoML simplifies ML processes, helping you build, train, and explain ML models without data movement or added costs. Similarly, in-database LLMs, a built-in vector store, and embedding models enable GenAI, semantic search, and retrieval-augmented generation (RAG) at reduced infrastructure costs and without data movement.
https://www.mysql.com/products/mysqlai/

As a developer, you need to clean what AI wrongfully believes is correct – often times going back to basic SQL. This is yet another reason why programming is fun.

So many databases, what to use and/or recommend?

As a purist, I would recommend PostgreSQL, which is the closest distribution to fully ANSI-complaint. Unfortunately the industry does not run on purists unless coding in Ruby on Rails (RoR) . The most common RoR installation is on the Heroku platform, which offers PostgreSQL as its default database. In any case, it is not heavily used as giants like Oracle or SQL Server (Microsoft) and this might make it more secure than the latter two – fewer hackers trying to break PostgreSQL or its security. Maybe this is a good reason to be open source as anyone can fix a bug before it gets maliciously exploited.

As a nostalgist, I would recommend Oracle, which is the system that I learned with. Unfortunately Oracle has somewhat of a bad reputation in the industry after taking over Sun. Some developers opted to quit rather than joining Oracle. The OpenOffice development group formed LibreOffice, while MySQL development group formed MariaDB. Oracle is proprietary (closed) source.

As a practicalist, I would recommend MariaDB and not MySQL to stay away from whatever toxic environment that has made many programmers leave Oracle. As a drop-in replacement of MySQL, MariaDB has taken over what had been the MySQL place in the LAMP (from Linux, Apache, MySQL/MariaDB, Perl/PHP/Python to Linux, Apache, MariaDB, Perl/PHP/Python) web stack maintaining the full open source mantra. Note that the web stack could also be referred to as WAMP when running on a Windows server.

LAMP is an Open Source Web development platform that uses Linux as the operating system, Apache as the Web server, MySQL as the relational database management system and PHP as the object-oriented scripting language. (Sometimes Perl or Python is used instead of PHP.)
https://www.techtarget.com/whatis/definition/LAMP-Linux-Apache-MySQL-PHP

As a realist, rather than teaching MariaDB, I would teach MySQL as it has a much stronger market share and more students might be interested in MySQL rather than its fork. In other words, new comers to the industry might have heard of MySQL, but not MariaDB. MySQL has a combination of open source (full open source for the Community Edition, which I am set to teach next) and proprietary (closed) sources.

As a minimalist, I would recommend SQLite (actually sqlite3) – no server, just data. SQLite is used in many systems including web browsers and mobile telephones. The only drawback is the lack of robust security or user authentication, but it is fully open source so its users clean up bugs right away.

Introduction to Structured Query Language for Data Analytics (SF25SQL6172025) — Day 2

We are getting data from a table.

  SELECT field1,
    field2,
    field3
  FROM table1;

We are getting data from tables.

  SELECT table1.common_field1, -- field/data on both tables
    table1.field2,
    table2.field3,
    table2.field4
  FROM table1 -- left table (main if `LEFT JOIN`)
  INNER|LEFT|RIGHT JOIN table2 -- right table (main if `RIGHT JOIN`)
    ON table1.common_field1 = table2.common_field1; -- relation between tables

Download the class notes for day 2.

Installing SQL Server Express in macOS #4

Free:

  1. Install Docker Desktop on Mac
    https://docs.docker.com/desktop/setup/install/mac-install/
  2. How To Install SQL Server on Mac (M1 | M2 | M3)
    https://www.youtube.com/watch?v=3BFxALltQaM
  3. DBeaver – Microsoft SQL Server
    https://dbeaver.com/docs/dbeaver/Database-driver-Microsoft-SQL-Server/

Paid:

  1. Buying Parallels
    https://www.parallels.com/
  2. Buying a copy of Windows
    https://www.microsoft.com/en-us/software-download/windows11

More information:

  1. Installing SQL Server Express in macOS
  2. Installing SQL Server Express in macOS #2
  3. Installing SQL Server Express in macOS #3

For reference purposes only, not responsible for external resources.

Importance of Learning SQL & Related Technologies

Many new programmers don’t dedicate time to learning SQL (regardless of vendor and its extensions) and/or understanding how data is stored, yet expect to call and write data using an external or third-party language.

As a programmer using data, you should know where that data lives (flat file, table, schema, database, cube, etc.) and how to access it (SELECT, INSERT, UPDATE, etc.). You should understand relational database concepts as well as the behavior of the database — characteristics given by the vendor (for example, auto-commit in Microsoft T-SQL or the need to use COMMIT in order to write data into a table in Oracle PL/SQL).

Some modern languages like Ruby can write SQL code for you in the back-end hidden from the programmer (for example, db/structure.sql when using Active Record, the default method in Ruby), but there’s no assurance that the code is correct regardless of language or implementation. Even if you decide to depend on the language environment to write the SQL code for you, you should be able to edit and improve the code as needed.

You must have full control of your data as well as your code. This is not limited to accessing data, but it also demands that you should control the security of such data in your program. If you can control data security in the database, it’s even better, but this is usually in the hands of the database administrator (DBA). Always think like a black hat hacker and understand how your data can be compromised and stolen. As much as you don’t want hackers to break your program, you don’t want them to break your data.

In other words, learn SQL and know your data.

Top 10 Best Programming Languages To Learn in 2024 — Superior Codelabs IT Services

A new list of hot languages is out — well one list of many.

Once again, Python is the hottest language, but what I find interesting is the inclusion of assembler although it does not specify which version (NASM, FASM, etc.).

  1. Python
  2. JavaScript
  3. Java
  4. C++
  5. Swift
  6. Go (Golang)
  7. Kotlin
  8. Rust
  9. TypeScript
  10. PHP
  11. Ruby
  12. Shell Scripting (Bash)
  13. SQL
  14. Scala
  15. C#
  16. MATLAB
  17. Dart
  18. Julia
  19. Assembler

This article also includes a quick explanation why all programmers should learn how to manage data.

How important is SQL for a programmer or developer?
SQL (Structured Query Language) is crucial for anyone working with databases. Whether you’re a developer, data analyst, or database administrator, understanding SQL is essential for managing, querying, and manipulating data in relational databases.
https://www.linkedin.com/pulse/top-10-best-programming-languages-learn-2024-superiorcodelabs-yx3ec/

This list and quote are neither an endorsement nor an advert.

In other words, these are languages recommended to learn in order to get a good paying job.