Lines Matching refs:Upper
4580 // 2. Subvector should be inserted in the upper part
4635 // Simple case when we put subvector in the upper part
4637 // Zero upper bits of the Vec
6212 /// the lower 128-bit of V0 and the upper 128-bit of V0. The second
6214 /// and the upper 128-bit of V1.
6221 /// dag node takes the upper 128-bit of V0 and the upper 128-bit of V1.
6228 /// the upper 128-bits of the result.
6761 // Build both the lower and upper subvector.
6764 SDValue Upper = DAG.getBuildVector(
6767 // Recreate the wider vector with the lower and upper part.
6769 return concat128BitVectors(Lower, Upper, VT, NumElems, DAG, dl);
6770 return concat256BitVectors(Lower, Upper, VT, NumElems, DAG, dl);
7929 // Upper half must be undefined.
7934 // Remainder of lower half result is zero and upper half is all undef.
8071 "Extension offset must be in the first lane or start an upper lane.");
8266 // upper lane.
8316 // MOVQ, copying the lower 64-bits and zero-extending to the upper 64-bits.
10876 // Upper half is undef and lower half is whole upper subvector.
10886 // Lower half is undef and upper half is whole lower subvector.
10910 // i.e. 0 = Lower V1, 1 = Upper V1, 2 = Lower V2, 3 = Upper V2.
11661 // Handle special cases where the lower or upper half is UNDEF.
12719 // upper bits of a vector.
12745 // the upper bits of a vector.
13530 // Zero out the upper parts of the register.
13991 // Use vpunpckhwd for 4 upper elements v8i16 -> v4i32.
13992 // Concat upper and lower parts.
13996 // Use vpunpckhdq for 4 upper elements v4i32 -> v2i64.
13997 // Concat upper and lower parts.
15531 // Make sure the lower and upper halves are both all-ones.
19069 // If the upper half of the input element is zero then add the halves'
19070 // leading zero counts together, otherwise just use the upper half's.
19079 // Check if the upper half of the input element is zero.
19084 // Move the upper/lower halves to the lower bits as we'll be extending to
19460 // logical shift down the upper half and pack back to i8.
19465 // and then ashr/lshr the upper bits down to the lower bits before multiply.
19551 // Multiply, lshr the upper 8bits to the lower 8bits of the lo/hi results and
19770 // Splat sign to upper i32 dst, and SRA upper i32 src to lower i32.
19771 SDValue Upper =
19776 Ex = DAG.getVectorShuffle(ExVT, dl, Upper, Lower, {5, 1, 7, 3});
19778 Ex = DAG.getVectorShuffle(ExVT, dl, Upper, Lower,
19781 // SRA upper i32, SHL whole i64 and select lower i32.
19782 SDValue Upper = getTargetVShiftByConstNode(X86ISD::VSRAI, dl, ExVT, Ex,
19788 Ex = DAG.getVectorShuffle(ExVT, dl, Upper, Lower, {4, 1, 6, 3});
19790 Ex = DAG.getVectorShuffle(ExVT, dl, Upper, Lower,
20209 // all lanes and the upper i64 is ignored. These shuffle masks
20324 // Logical shift the result back to the lower byte, leaving a zero upper
24877 // PSHUFHW: permute upper 4 elements only.
25468 // matter. Check that the upper masks are repeats and remove them.
25716 // moves upper half elements into the lower half part. For example:
