Document psycopg concepts and features in prose

[kpinc]

Hello,

Add a section to the documentation that introduces psycopg features and vocabulary in narrative prose. Nothing wrong with what's there, which introduces by way of dissecting an example. But sometimes narration gives a better overview. Different people have different learning styles (and "the php way" of documenting by example drives me nuts!)

This patch puts each sentence on a new line, in order to make future patches easier to scan.

Some of the index entries may seem to show up too early in the prose, but I believe that they catch the earliest mention of the indexed terms.

I'm not really familiar with the content of the index, so some of the index entry pairs might be able to be improved.

I have also introduced some new comments into the example code presented in the "Basic Module Usage". There is little that is not explained in the subsequent text, where the code is dissected. But I think it's important, because people don't read. Especially people who look at examples for understanding. I suppose this part could be a separate patch. But some of the narrative I've added explicitly says "go look at the example below", and without some of the new code comments the example appears to contradict the new narrative.

Started looking at documenting how literal strings interact with execute() and wound up writing this instead. (!)

P.S. I don't like the "Transactions management" index entry, or section title. I believe both should be "Transaction management". But that's for somebody else to fix.

[dvarrazzo]

Hello,

thank you very much, I am a fan of an introductive section.

The fact that you describe it to be "in prose" suggests that the rest of the documentation is written in poetry, which would actually be interesting, but maybe for a future project.

The index is this. It's worth checking that the changes you are making are the effect you want to have. I am not sure it does: I don't think you can add index entries if not on sections titles? It's worth checking before you add material that either doesn't look good or it causes us feeling the need to maintain without actually doing anything.

I haven't read the content you wrote, just skimmed it. I think it has some imprecision and I think it jumps ahead too quickly to advanced topics (e.g. mentioning pipeline mode before mentioning the cursors). I don't have time for a thorough review now: it will give it a proper read later on.

Maybe you can limit the content of your Concepts section to the basic features, and add a sort of "there's more to come" section explaining roughly problems (latency, types adaptation) and act as a teaser jumping to different "prose" sections in other parts of the documentation where you can introduce more concepts and relative features. This page, for instance, is pretty much just the ToC, whereas it could have an introductory section.

In these sections you could go fancy with "warning", "note", "see also" blocks to highlight the "jump board" nature of that section.

Feel free to change transactions -> transaction management if you think it is appropriate (I don't know if English is your primary language. It isn't mine, and it isn't the one of other major contributors, so it's not surprising if you stumble in less than idiomatic sentences every now and then. Or in pieces of poetry, too).

Thank you very much!

[kpinc]

Hi Daniele, I thought it best to reply before you did the work of a complete review.

On Sun, 29 Jan 2023 00:42:32 -0800 Daniele Varrazzo ***@***.***> wrote: thank you very much

You're welcome.

The fact that you describe it to be "in prose" suggests that the rest of the documentation is written in poetry, which would actually be interesting, but maybe for a future project.

ChatGPT anyone? ;-) "Narrative" would probably have been a better choice. But I don't try too hard except when writing important stuff. :)

[The index is this](https://www.psycopg.org/psycopg3/docs/genindex.html). It's worth checking that the changes you are making are the effect you want to have. I am not sure it does: I don't think you can add index entries if not on sections titles?

You can put index entries anywhere. Even in the middle of paragraphs, but that seems a little counter-productive. (Well, I don't know if an index entry will break a block of code into 2 pieces.)

It's worth checking before you add material that either doesn't look good or it causes us feeling the need to maintain without actually doing anything.

I did check that my entries were ending up in the index, and looking like what I expected. But I didn't read the whole index to see if there's an additional term I should be indexing, or if one of the terms I indexed should really be a different term that's already indexed.

I haven't read the content you wrote, just skimmed it. I think it has some imprecision

I tried to be precise, and accurate, but.... I did work hard to ensure each sentence said what I wanted it to. But maybe my understanding is flawed. Or maybe I just failed with some of them.

and I think it jumps ahead too quickly to advanced topics

It's not really supposed to be a "how to". It's supposed to touch all the "big parts" and provide an overview. I tried to provide all sorts of links, from those a beginner might need, like the SQL link to the wikibook on SQL, to the advanced topics. (The front of the wikibook on SQL was the only web page I found that clearly and succinctly described SQL.) Hopefully the paragraphs are written properly (topic sentences, etc.) so that a beginner can tell that they don't really need to care about what's in some of them and can skip forward without necessarily trying to grasp everything and reach full understanding. I do think that starting "at the beginning", with connections, draws the reader in. I did have a 14 year old read it, as a test. I passed, but one never knows if that's because of lack of real interest on the part of the 14 year old. :-) See below for other ideas regards organization.

(e.g. mentioning pipeline mode before mentioning the cursors).

I wanted to mention all the performance optimizations "together", and had already talked some about performance in the previous paragraph. I'd written the index entries and realized I'd left off mention of some big parts of Psycopg and was looking for a place to put them and ....

Maybe you can limit the content of your Concepts section to the basic features, and add a sort of "there's more to come" section explaining roughly problems (latency, types adaptation) and act as a teaser jumping to different "prose" sections in other parts of the documentation where you can introduce more concepts and relative features. [This page](https://www.psycopg.org/psycopg3/docs/advanced/index.html), for instance, is pretty much just the ToC, whereas it could have an introductory section.

To my mind that would defeat one of the main goals. I wanted one section that provides a high-level overview of "everything". Or at least everything "big". That way people don't have to read the whole document or figure out what links to click on to know that they've an awareness of what they might want to think about. How does a reader unfamiliar with the topic know when they're "done" with a rough grasp of a subject? Without reading the entire document. IMO this is an important feature in documentation and difficult or impossible when the author has not written a contiguous block and labeled it "overview". (If you really want to be picky, transactions are not "basic". The reader could probably benefit if transaction related content in the current "basic" section was separated out from query related content. That way people could do basic read operations, which might cover a lot of use-cases, and then go back and learn write operations. Or "the basics" could cover write operations too, but not transactions. After all, if the reader does use context managers as in the example then a simple program will work -- data modifications in the db will be preserved. I don't think your organization is a big issue. You too were writing a single "learn what you really need to know" section. ;-) If you don't like the new text in the "Basic" top-level section, perhaps it belongs in a top-level section of it's own? Maybe the title is not good, and it should just be "Overview"? I don't think the title is bad, but improvements are likely possible.

Feel free to change transactions -> transaction management if you think it is appropriate

Ok.

(I don't know if English is your primary language.

English is my native language. But writing, especially writing sentences, is hard. :-)

It isn't mine, and it isn't the one of other major contributors, so it's not surprising if you stumble in less than idiomatic sentences every now and then. Or in pieces of poetry, too).

:-) Some of the section on transactions is a bit awkward. That's where I primarily noticed problems with English. FWIW.

Thank you very much!

You're welcome. I'll try to get to the "literal string" doc patch, at some point. Regards, Karl ***@***.***> Free Software: "You don't pay back, you pay forward." -- Robert A. Heinlein

[kpinc]

On Sun, 29 Jan 2023 07:27:18 -0600 "Karl O. Pinc" ***@***.***> wrote:

On Sun, 29 Jan 2023 00:42:32 -0800 Daniele Varrazzo ***@***.***> wrote: > and I think it jumps ahead too quickly to advanced > topics It's not really supposed to be a "how to". It's supposed to touch all the "big parts" and provide an overview. > Maybe you can limit the content of your Concepts section to the > basic features, and add a sort of "there's more to come" section > explaining roughly problems (latency, types adaptation) and act as > a teaser jumping to different "prose" sections in other parts of the > documentation where you can introduce more concepts and relative > features. <snip> To my mind that would defeat one of the main goals. I wanted one section that provides a high-level overview of "everything". Or at least everything "big". If you don't like the new text in the "Basic" top-level section, perhaps it belongs in a top-level section of it's own?

I've submitted a new patch, moving the new section into 2nd level section of it's own under "Getting Started with Psycopg 3". I've added to the top of the new section, telling the reader what they will find in the section and giving them guidance as to the section's purpose. The text I've added is to tell the reader why they want to read the (sub)section, and what to expect when reading the section. This is supposed to orient the reader and make reading easier. In short, I've split the version of the "Basic Usage" section you saw in my last patch, into 2 sections. And guided the reader to assist them in determining which, or which parts of each, they might want to read.

> Feel free to change transactions -> transaction management if you > think it is appropriate Ok.

Done.

Some of the section on transactions is a bit awkward.

I've been bold. In the "Basic Usage" section I've re-worked some of the transaction related docs. (I'm sure there's more work to be done in transactions.rst, but not for this patch.) I've also extensively added to the top of the "Basic Usage" section, and re-titled it. The additions tell the reader what they can expect to find in the section. Some sub-section titles have also been altered. I hope to address many of your concerns by way of the reader guidance I've added to the start of the sections. I moved the Python DB-API sub-section from the "Basic Usage" section into the new section, and twiddled its content. The "Installation" section needed minor adjustment related to the embedded links to the "Basic Usage" section. Regards, Karl ***@***.***> Free Software: "You don't pay back, you pay forward." -- Robert A. Heinlein P.S. It seems to take some time to pass the github CI tests. It makes me wonder if documention-only changes need more than the lint test.

[kpinc]

Hi, I understand that the changes in this PR are already large. Regardless, I want to offer to re-work the text above the first block of example code appearing in transactions.rst. And add those changes to this patch. I'm interested in doing the work while I have momentum and in order to make all the transaction verbiage consistent. (Some of what is written there is wrong, or at least wrong some of the time. Not right, and not entirely accurate.) Let me know your thoughts. Regards, Karl ***@***.***> Free Software: "You don't pay back, you pay forward." -- Robert A. Heinlein

[dlax]

Haven't read all changes either (they're big!), but, if we are to rework the documentation structure, I wonder if we shouldn't take inspiration from "well-established" practices, like what's theorized by https://diataxis.fr/ (see projects following this practice, e.g. numpy).

[dvarrazzo]

@dlax I am not aware of that resource, will read it.

@kpinc will review your changes in the coming days.

Thank you very much!

[kpinc]

On Tue, 31 Jan 2023 08:46:39 -0800 Daniele Varrazzo ***@***.***> wrote: @dlax I am not aware of that resource, will read it. @kpinc will review your changes in the coming days. Thank you very much!

Don't thank me yet. What I've written is not exactly right. And I think I can improve on "concepts" a bit. I think I'm going to have to read the code or do some other research to figure out exactly what psycopg is doing. For example, I don't believe that, by default, a transaction is started when data content in the db is modified. I've a feeling that a transaction is started when any SQL at all is sent to a as-yet unused connection. By default that is. Once I figure out exactly what's happening I probably won't resist a separate pull request that re-works transactions.rst. I kind of like to write as I go to track my understanding. Anyway, I would like an overall review from you. To see if you like the overall approach or anything else you'd like to say. Regards, Karl ***@***.***> Free Software: "You don't pay back, you pay forward." -- Robert A. Heinlein

[kpinc]

On Tue, 31 Jan 2023 07:01:19 -0800 Denis Laxalde ***@***.***> wrote: Haven't read all changes either (they're big!), but, if we are to rework the documentation structure, I wonder if we shouldn't take inspiration from "well-established" practices, like what's theorized by https://diataxis.fr/ (see [projects following this practice](https://diataxis.fr/adoption/), e.g. [numpy](https://numpy.org/doc/stable/)).

Thanks. I'll take a look. Regards, Karl ***@***.***> Free Software: "You don't pay back, you pay forward." -- Robert A. Heinlein

[pauloxnet]

… if we are to rework the documentation structure, I wonder if we shouldn't take inspiration from "well-established" practices, like what's theorized by https://diataxis.fr/ …

I can't agree more.
The Daniele Procida's project is excellent and I highly recommend it's workshop
https://youtu.be/710uQqIqsWk