We need a new operator '..' to access to members of childitems of a collection

[nutix56]

#example 1
$files = Get-ChildItem -File
#Let's get the length of every file.
$files..Length

#example 2
$hash = @{
    first = @{
        name = 'one'
        value = 1
    }
    second = 'two'
    'first name' = 'Jobs'
}
#The member intellisense would be clearer in PowerShell_ISE editor and the TAB completion.
$hash..first

[mklement0]

If I understand you correctly, you want a dedicated operator named .. that makes member enumeration explicit and thereby avoids the ambiguity with the current .-based member enumeration, where a property of a given name at the collection level takes precedence over element-level properties (such as .Length in your first example).

I don't quite understand how your 2nd example fits into this, however. Can you clarify by updating the original post, and, while you're at it, use code blocks (```powershell ...) to better format your code?

Having a dedicated operator would potentially allows us to solve 3 existing problems:

• as stated, it would resolve the ambiguity of .: the new operator signals the explicit intent to operate on the elements of the collection, not the collection itself.

• it would allow us to provide an alternative to the pipeline-like behavior of . (implicit unwrapping, flattening of arrays) - see Member-access enumeration should not apply pipeline enumeration logic #6802 (though perhaps having different behaviors would ultimately be too confusing)

• it would allow implementing assignment operations too, which are currently not supported for fear of unintended use - see Member-access enumeration works with setter methods, but not with property assignments #5271 and, in particular, Member-access enumeration works with setter methods, but not with property assignments #5271 (comment)

[RichardSiddaway]

How does Powershell differentiate .. when used as the range operator from .. when used to access the members of a collection.

not arguing against the need to access the collection member's properties just don't understand how the two uses of the same operator symbol will work

[mklement0]

@RichardSiddaway:

I was focusing on making the case for such an operator in the abstract, but you're right, the syntactic form needs careful consideration too.

.. is problematic for its potential to be confused with the range operator alone.

Groovy uses *., for instance, and at first blush that may work, given that we don't allow unquoted property names to start with digits (unless it's an all-digit name, in which quoting or use of an expression could disambiguate - see #6946).

But let's see if there is a fundamental willingness to introduce such an operator.

[nutix56]

Sorry, my english is poor!
Yes, that's what i mean, thanks @mklement0
About the confusing you mentioned, the range operator operates on two variables or literal value, like:

1..10

or（PowerShell 6.0+)

'a'..'z'

or

$from..$to

and their mixed mode combination.

But when we use the new operator as i said, it's like:

$files..Length

the second operand is a member name, it's not a variable which must starts with '$'.
Even in the following case:

function one { 1 }
#3..one is error code
#we must write like this
3..(one)
#or 
3..$(one)

The second example is intend to use items of IDictionary object as elements of collection object, just like the 1st example. It's a extensional advice.

[mklement0]

Thanks for updating the initial post, @nutix56.

Re the 2nd example: I'm still not entirely clear what you expect the output of $hash..first to be: The key-value pairs of the nested hashtable (2 [System.Collections.DictionaryEntry] instances representing the name and value entries each)? Or just their values?

As for the syntax: I meant confusing to the user, not necessarily whether it technically works (syntactically):

• In terms of being confusing, perhaps the fact that ranges can start with a variable, but rarely do, makes that less of a concern.

• Even technically, though, there's ambiguity that would have to be resolved with non-obvious rules (and I can't speak to how much it would complicate parsing):

PowerShell supports expressions as property names, so $files..Length could be written as $b='Length'; $files..$b (at least it wouldn't be a good idea to disallow this with .., given that it's supported with .). So with a token such as $a..$b you would then have to infer from the values of $a and $b whether they're both numeric (-> range operator) or not (-> member enumeration). Note that the range operator is extremely flexible in what it accepts as numeric input, so that ' 5 '..$False is equivalent to 5...0, for instance).

[msftrncs]

The first issue example 1, work-around is $files | select-object Length, didn't realize listing all the file lengths was so difficult. This seems peculiar, since in a hash table, the keys overload the base properties, but here a property is getting hidden by a base PowerShell property. The work around for the hash is to use the method call get_[propertyname]()

The second example I don't think shows anything. There is no 'first' property that is getting overloaded. $hash['first'] is also acceptable. I do think this should be the correct way to retrieve values from the hash, however.

[mklement0]

Thanks, @msftrncs.

There's a more concise workaround: (Get-ChildItem).ForEach('Length') (and you'd need to add -ExpandProperty to yours).

Interesting that the logic is reversed with hashtables (entry keys shadowing the hashtable's own properties when using dot notation).

Unfortunately, that precludes meaningful application of the potential new operator to hashtables, because . already acts like that operator.

If the logic weren't reversed, that is, if the behavior were consistent with member enumeration, @{ Count = 'Chocula' }.Count would return 1 (it currently returns Chocula), i.e., the hashtable's own property, whereas @{ Count = 'Chocula' }*.Count would then unambiguously target the entry, returning Chocula.

[BrucePay]

@mklement0 This is an aside, but WRT to assigning to a property on each member of a collection, you can do it with .foreach() as follows:

PS[2] (8) > $l = @{a=1},@{a=2},@{a=3}
PS[2] (9) > $l.foreach("a", 13)
PS[2] (10) > $l

Name                           Value
----                           -----
a                              13
a                              13
a                              13

Now regarding what operator we might use (since '..' is definitly out for the reasons stated) Groovy uses '*' for spreading (=== "splatting") whereas PowerShell uses '@'. Consequently the corresponding operator in PowerShell would be "@."as in:

 $lengths = $list@.Length

Now in a previous language I designed, I used '->' for a similar purpose (but slightly different semantics)

$lengths = $list->Length

I liked the fact that it was clearly visible in the code. Another possibility is to have a semantic such that if the RHS is a scriptblock, then it gets applied to each element, minifying the foreach scenario even further.

(1..10)->{$_ * 2}

(Early in PowerShell we spent a bunch of time talking to the folks in the languages group in MS Research. One of the researchers was very keen on having a minimal foreach/map/apply/select notation so this would probably make him happy :-)

[mklement0]

Thanks, @BrucePay.

Good point about @. being more appropriate.

Thanks for the reminder about .ForEach() also being able to set properties.

Though .ForEach() therefore technically gives us everything we'd need the new operator for, I still think the latter is a useful addition (and it doesn't sound like you disagree, I'm just spelling it out), because:

• with .ForEach() you need to quote the property name, and you need method syntax.

• the fact that .ForEach() even supports property extraction by a string as the name and especially assignment is not intuitively obvious and easy to forget - probably also because the use case of passing a single script block, as you would to ForEach-Object and the foreach loop, is so prevalent.

• @. is much more concise, and concision is important for frequently used core features

that if the RHS is a scriptblock
(1..10)->{$_ * 2}

In the case of a script block, however, I feel that .ForEach() is more appropriate (not least because there wouldn't be a . analog for something (1..10)@.{$_ * 2}):

(1..10).ForEach({$_ * 2}) # I know you can omit the (...), but that seems fraught to me.

Though, as I've proposed, I wish we had .ForEach() in operator form:

(1..10) -foreach {$_ * 2}

@chuanjiao10:

Yes, getting carried away with abstract symbols is generally problematic, and there is an inherent tension between readability and concision.

However, if the . operator (member access) makes sense to you, then @. should make sense to you as a natural extension of . for an inherently collection-focused language such as PowerShell.

The use of symbols is less problematic for frequently used features, because seeing them often makes them second nature.

Here, @. creates an operator whose @ symbol is already established as pertaining to collections, as Bruce states (argument splatting, @(...)), which helps with remembering its purpose.

[felixfbecker]

I don't see the need for this. I find it rare that a property conflicts with a collection property, and for the other cases, you can easily do $collection |% property. What's better about $collection..property?

[mklement0]

As an aside: I suggest we use @. in the discussion from now on.

What's better about $collection@.property

• No situational ambiguity, especially with property names provided a variables or expressions.

• Better performance, especially compared to ... | % property, but even relative to .ForEach( 'property')

• Conceptual clarity: you signal the intent to perform member enumeration as opposed to accessing a property of the collection itself.

• Concision

[felixfbecker]

All your points except your second one can be made for |% property too.

[mklement0]

By that logic, member enumeration via . should never have implemented - yet it became a very popular feature.

The second point, performance, matters - member enumeration is fast, and you don't want to sacrifice speed just to work around its limitations.

Additionally, re concision: Use of aliases is discouraged in scripts, so your alternative would really be ... | ForEach-Object property.

[felixfbecker]

. already existed to access properties, so there was no additional operator introduced. It's just PowerShell being a bit smarter in certain situations to do what you probably meant to do. It makes sense, because PowerShell is smart enough to wrap and unwrap values and collections in other contexts as well (e.g. when passing string to a string[] parameter, or a single-element array to a string parameter).

But whenever a new operator is introduced you have to consider that this is one additional thing for people to learn, one more thing that someone might not understand when they read a PowerShell script.

[mklement0]

. already existed to access properties, so there was no additional operator introduced.

Yes, but one should have been introduced.

As great a feature as member enumeration was to introduce in principle, conflating it with the regular member accessor, ., resulted in the problematic behavior that @. aims to fix.

[Jaykul]

Although that is also technically valid with numbers (it's not very often we do fractional modulo):

$ten = 10; $ten%.2

PowerShell won't allow a .$variable (or .text) on the right-hand side of the modulo operator, so I think it's safe enough...

[nutix56]

The operator focus on:

But, maybe it's better to use '...'

We can use it as a replacement for the simplest loop statement, like

$lenths = $(foreach($f in Get-ChildItem -File) { $f.Length})
$lenths

=>

$lengths = (Get-ChildItem -File)...Length

'...' means more enumerable elements,

when we use it between a collection object and its element property name, it means enumerate the property on each element of the collection object.

I think it's very informal.

[RichardSiddaway]

If all you want is a single property why not use select-object

PS> Get-ChildItem -File | select -ExpandProperty length

[nutix56]

If all you want is a single property why not use select-object

PS> Get-ChildItem -File | select -ExpandProperty length

It's more convenient, doesn't it?

[mklement0]

To recap the issue succinctly:

Member enumeration with . is:

• (a) delightfully concise
• (b) fast

Its only drawback is the inability to predictably target the elements' members rather than the collection object's in case of naming conflicts.

Having a distinct syntax - next to . - that unambiguously targets the elements would solve that problem (and would also allow addressing other . limitations - see above).

The current workarounds fall short:

• Select-Object -ExpandProperty length satisfies neither (a) nor (b): it is verbose and slow.

• .ForEach('Length') somewhat satisfies (b) (still slower than member enumeration), but not (a)

To my mind, the best candidate for the new syntax, after the discussion above, is %.:

$collection%.Length # Return the *elements'* .Length property values.

[Jaykul]

I'm not sure we need it (since collections don't have many members), but that syntax seems clean, concise, and unambiguous.

[msftrncs]

Technically, isn't $collection%.property then the same as $collection.foreach{$_.property}? (but shorter)

[mklement0]

@msftrncs: yes, though that is somewhat slower - and more verbose - than the aforementioned $collection.ForEach('Property') syntax (which is still verbose) - the latter syntax also supports assigning to property values more efficiently (pass an extra argument with the value).

Speaking of which: %., with its unambiguous intent, could and should support assigning to properties as well (which with . was deemed too dangerous - see #5271 (comment)); e.g., you could do something like:

# Assign the - invariably *same* - property value to all elements of the array.
(Get-Item *.txt)%.LastWriteTime = Get-Date

@Jaykul, glad to hear you like the syntax; re the need for it:

• PowerShell treats collections uniformly in many contexts (which is a good thing), and you may not even know what particular collection type you're dealing with in a given situation (not least because PowerShell itself returns different collection types situationally ([object[]] vs. [System.Collections.ArrayList] vs. [System.Collections.ObjectModel.Collection[PSObject]]).

  • That is, in a given situation you may not even be aware of the potential for name conflicts, and you won't find out until later.

• As for specific members:

  • The Item method present on most collections is pitfall when dealing with XML or JSON data with such element / property names, for instance; Length and Count to lesser degrees.

  • ToString is currently invariably invoked on the collection itself (where it is more or less useless), but having a concise, predictable way to call the ToString method on all elements of a collection would be helpful, so that you could do the following (which with just . currently fails):

# Predictably call .ToString() on the elements, not the collection itself.
((Get-date), (Get-Date).AddDays(1))%.ToString('dd')

# The above would be the concise equivalent of:
((Get-date), (Get-Date).AddDays(1)).ForEach('ToString', 'dd')

[Jaykul]

Yes, but I don't find this argument compelling at all:

In a given situation you may not even be aware of the potential for name conflicts, and you won't find out until later.

I mean, that's obviously true. But most of the time, it doesn't matter. It's not that this is non-deterministic and might be different on different computers, so I can just write it and run it.

Adding a longer syntax that's more explicit about what it's doing and therefore theoretically clearer about what will happen doesn't mean that anyone's going to use it, because while it's technically clearer, we can't go back in time and do this unambiguously (i.e. we won't remove the fact that . unrolls child properties as long as they don't conflict with properties on the collection).

So this will be a great in those specific but rare situations where we actually run into a duplicate -- but people will have to actually know about it, and be 100% sure they don't need their code to run in PowerShell 7 or earlier.

Potentially this would just add an optional operator that's confusingly similar to several other operators and practically not used in the real world, and is therefore always confusing when people see it.

[mklement0]

But most of the time, it doesn't matter.
So this will be a great in those specific but rare situations where we actually run into a duplicate

The point is: You shouldn't have to worry about when it does matter.

When you write code, your intent is always clear: you'll know ahead of time whether you're looking for a property of the collection or properties of its elements.

Currently, if you're not careful (and remember all member names of the collection you're dealing with), that intent can be thwarted in the latter case, with potentially confusing symptoms.

Having a direct, unambiguous expression of one's intent in the form of %. is therefore a great asset in my book.

Yes, the behavior of . can't be changed anymore, but anyone who understands what %. offers may consider adopting it as a matter of habit.

people will have to actually know about it, and be 100% sure they don't need their code to run in PowerShell 7 or earlier.

That applies to any new feature; it's the price of progress.
We have the same problem with the ternary operator, null-coalescing, pipeline chains...

As for the concern about adding another, similar-looking operator: Let's make it easy to look up help for operators: #11339

[Jaykul]

The problem is that this isn't true:

When you write code, your intent is always clear: you'll know ahead of time whether you're looking for a property of the collection or properties of its elements.

Even if we add this operator, only half of that is possible: you'll be able to specify that you're looking for a property of the elements. That's it. When you're looking at code that uses %. you'll know the author meant a property of the elements. The other half isn't possible -- when you're looking at code, you won't know for sure that they deliberately chose to use . instead of %. because they're looking for a property of the collection...

If we do add this feature, we should also add a script analyzer rule that recommends people use %. whenever the analyzer can tell that they're addressing the elements of the collection. It would be a hard rule to write, because you don't know for sure whether a command returns multiple items or not, which is one of the reasons we got the automatic unrolling in the first place -- so we could just write it without worrying about it 😉

[mklement0]

When you're looking at code that uses %. you'll know the author meant a property of the elements.

That's all I ask for. You can't fix the past, but you can make things better in the future.
That is, going forward you'll be able to - and should - express your intent unambiguously, and the reward is predictable results.

we should also add a script analyzer rule that recommends people use %. whenever the analyzer can tell that they're addressing the elements of the collection.

Great, if feasible - but it sounds like it isn't.

one of the reasons we got the automatic unrolling in the first place -- so we could just write it without worrying about it 😉

That would be great if you didn't have to worry about it, but the whole reason for this discussion is that you currently do (situationally, and you may not anticipate when it'll hit you), and %. offers a way so you no longer have to.

[iRon7]

I don't see the need for this. I find it rare that a property conflicts with a collection property, and for the other cases, you can easily do $collection |% property. What's better about $collection..property?

To compliment @mklement0 arguments:

• I don't think it is rare, in fact, due to the (common and useful) ToString method conflict, I even thought for a long time that member enumeration was only applicable to properties.

• $collection |% property often requires an extra pair of parenthesizes when combined with other statements (as other chained member enumerations or concatenations), where a statement as $collection..property doesn't.

My thoughts: it doesn't have to be a shorthand syntax, it just need to be consistent and easy readable. Why not adding a PowerShell property like member to each collection (e.g. using the ForEach method also as a property,-if possible-) that holds a bare minimum of members, knowing that currently e.g. a simple array already contains ((,@() | gm -Force).Count) 49 members (excluding Count, Length, GetType(), ToString(), etc.).

Wishful thinking:
Currently: ($collection |% GetType).Name ,
Purposed: $Collection.ForEach.GetType().Name

Currently: Write-host 'Week days:' ($DateTimes |% ToString('dddd')),
Purposed: Write-host 'Week days:' $DateTimes.ForEach.ToString('dddd')
(or even Write-host 'Week days:' $DateTimes.Format('dddd'), see: #14372)

$a |% {$_ |% Length} # Recursive collection
Currently: $a |% {$_ |% Length}
Purposed: $a.ForEach.ForEach.Length

[mklement0]

@iRon7, with the caveat that you need to specify the member name as a quoted string, the .ForEach() method already allows you to do this:

$dates = , (Get-Date) * 3

$dates.ForEach('GetType').Name

$dates.ForEach('ToString', 'dddd')

While your proposal of a .ForEach property lessens the syntactic awkwardness, a %. operator seems much preferable to me:

• Unequivocally operating on the elements of a collection will be a frequently employed feature, so concision is important.

• As for obscurity concerns: % is well established as having the connotation of "for each" (albeit in a command context, and in expression contexts % is the modulo operators, but combined with . I wouldn't expect confusion).

• Generally speaking,

  • We're talking about a general-purpose operation applied to arbitrary objects, so an operator is conceptually more appropriate than introducing another intrinsic pseudo type member; intrinsic members (and the ETS) are a wonderful thing, but we should strive to minimize the "magic" we superimpose across all .NET types, because it can cause confusion over which parts are native, and it can also causes naming conflicts, as is already the case with System.Collections.Generic.List`1.ForEach.

  • On a related note: The syntactic awkwardness of the .ForEach() and .Where() methods - which I presume is in part responsible for their relative obscurity - led me to propose the introduction of -foreach and -where operators a while ago, but the proposal was rejected - see https://github.com/PowerShell/PowerShell-RFC/blob/00aec722fe23d124e7fb7542e710491edf644cc0/X-Rejected/RFC0034-Expose-ForEach-Where-Methods-as-Operators.md

GitHub

PowerShell/PowerShell-RFC

RFC (Request for Comments) documents for community feedback on design changes and improvements to PowerShell ecosystem - PowerShell/PowerShell-RFC

[mklement0]

P.S., @iRon7: having a member do double duty as both a method and a property would also be confusing and introduce an inconsistency, because invoking method members without () should output the method signatures (overloads) - e.g., 'foo'.ToString. Curiously, this doesn't currently work for .ForEach and .Where, but that should be considered a bug - see #14376.