Lines Matching refs:Shuffle
3265 default: llvm_unreachable("Unknown x86 shuffle node");
3277 default: llvm_unreachable("Unknown x86 shuffle node");
3291 default: llvm_unreachable("Unknown x86 shuffle node");
3303 default: llvm_unreachable("Unknown x86 shuffle node");
3532 /// isPSHUFDMask - Return true if the node specifies a shuffle of elements that
3543 /// isPSHUFHWMask - Return true if the node specifies a shuffle of elements that
3572 /// isPSHUFLWMask - Return true if the node specifies a shuffle of elements that
3601 /// isPALIGNRMask - Return true if the node specifies a shuffle of elements that
3674 /// CommuteVectorShuffleMask - Change values in a shuffle permute mask assuming
3690 /// specifies a shuffle of elements that is suitable for input to 128/256-bit
3745 /// specifies a shuffle of elements that is suitable for input to MOVHLPS.
3781 /// specifies a shuffle of elements that is suitable for input to MOVLP{S|D}.
3803 /// specifies a shuffle of elements that is suitable for input to MOVLHPS.
3872 /// specifies a shuffle of elements that is suitable for input to UNPCKL.
3909 /// specifies a shuffle of elements that is suitable for input to UNPCKH.
4017 /// specifies a shuffle of elements that is suitable for input to MOVSS,
4039 /// shuffle bellow:
4047 // The shuffle result is divided into half A and half B. In total the two
4072 /// getShuffleVPERM2X128Immediate - Return the appropriate immediate to shuffle
4097 /// specifies a shuffle of elements that is suitable for input to VPERMILPD*.
4131 /// isCommutedMOVLMask - Returns true if the shuffle mask is except the reverse
4156 /// specifies a shuffle of elements that is suitable for input to MOVSHDUP.
4179 /// specifies a shuffle of elements that is suitable for input to MOVSLDUP.
4202 /// specifies a shuffle of elements that is suitable for input to 256-bit
4222 /// specifies a shuffle of elements that is suitable for input to 128-bit
4291 /// getShuffleSHUFImmediate - Return the appropriate immediate to shuffle
4322 /// getShufflePSHUFHWImmediate - Return the appropriate immediate to shuffle
4346 /// getShufflePSHUFLWImmediate - Return the appropriate immediate to shuffle
4370 /// getShufflePALIGNRImmediate - Return the appropriate immediate to shuffle
4452 /// getShuffleCLImmediate - Return the appropriate immediate to shuffle
4742 // a generic shuffle instruction because the target has no such instructions.
4809 // All i16 and i8 vector types can't be used directly by a generic shuffle
4819 // to use VPERM* to shuffle the vectors
4828 /// vector of zero or undef vector. This produces a shuffle where the low
4830 /// Idx. This produces a shuffle mask like 4,1,2,3 (idx=0) or 0,1,2,4 (idx=3).
4846 /// getTargetShuffleMask - Calculates the shuffle mask corresponding to the
4921 default: llvm_unreachable("unknown target shuffle node");
4928 /// element of the result of the vector shuffle.
4992 /// shuffle operation which come from a consecutively from a zero. The
5017 /// isShuffleMaskConsecutive - Check if the shuffle mask indicies [MaskI, MaskE)
5047 /// isVectorShiftRight - Returns true if the shuffle can be implemented as a
5081 /// isVectorShiftLeft - Returns true if the shuffle can be implemented as a
5115 /// isVectorShift - Returns true if the shuffle can be implemented as a
5232 // the shuffle mask.
5397 /// 2. A splat shuffle which uses a scalar_to_vector node which comes from
5579 // Quit if more than 2 vectors to shuffle
5779 // a constant pool load than it is to do a movd + shuffle.
5789 // convert it to a vector with movd (S2V+shuffle to zero extend).
5809 // a vector, we can do this with SCALAR_TO_VECTOR + shuffle of zero into
5860 // movd/movss) to move this into the low element, then shuffle it into
5865 // Turn it into a shuffle of zero and zero-extended scalar to vector.
5874 // Splat is obviously ok. Let legalizer expand it to a shuffle.
5877 // Instead of a shuffle like this:
5878 // shuffle (scalar_to_vector (load (ptr + 4))), undef, <0, 0, 0, 0>
5880 // shuffle (vload ptr)), undef, <1, 1, 1, 1>
5987 // Check for a build vector from mostly shuffle plus few inserting.
6026 // then it is safe to just drop this shuffle: V[i] is already in the
6069 // Try to lower a shuffle node into a simple blend instruction.
6179 // of the two input vectors, shuffle them into one input vector so only a
6195 // If BestLoQuad or BestHiQuad are set, shuffle the quads together and update
6196 // the shuffle mask. If a quad is scored as -1, that means that it contains
6210 // source words for the shuffle, to aid later transformations.
6242 // pshufhw, that's as cheap as it gets. Return the new shuffle.
6268 // shuffle mask element to zero out elements that come from V2 in the V1
6286 // Calculate the shuffle mask for the second input, shuffle it, and
6384 // 3. [all] v8i16 shuffle + N x pextrw + rotate + pinsrw
6429 // Calculate the shuffle mask for the second input, shuffle it, and
6559 /// done when every pair / quad of shuffle mask elements point to elements in
6636 /// which could not be matched by any known target speficic shuffle
6656 // Build a shuffle mask for the output, discovering on the fly which
6657 // input vectors to use as shuffle operands (recorded in InputUsed).
6658 // If building a suitable shuffle vector proves too hard, then bail
6678 // Find or create a shuffle vector operand to hold this input.
6693 // shuffle vector. Insert all elements into a BUILD_VECTOR instead.
6698 // Add the mask index for the new shuffle vector.
6738 // At least one input vector was used. Create a new shuffle vector.
6750 /// 4 elements, and match them with several different shuffle types.
6787 // implemented with two shuffles. First shuffle gather the elements.
6788 // The second shuffle, which takes the first shuffle as both of its
6853 // Break it into (shuffle shuffle_hi, shuffle_lo).
6923 assert(VT != MVT::v2i64 && "unsupported shuffle type");
6942 "unsupported shuffle type");
6990 // this is horrible, but will stay like this until we move all shuffle
7001 assert(VT != MVT::v4i32 && "unsupported shuffle type");
7008 // Reduce a vector shuffle to zext.
7044 // Check the shuffle mask.
7062 // Simplify the operand as it's prepared to be fed into shuffle.
7121 // If the shuffle can be profitably rewritten as a narrower shuffle, then
7178 assert(!V1IsUndef && "Op 1 of shuffle should not be undef");
7180 // Vector shuffle lowering takes 3 steps:
7184 // 2) Matching of shuffles with known shuffle masks to x86 target specific
7185 // shuffle nodes.
7186 // 3) Rewriting of unmatched masks into new generic shuffle operations,
7187 // so the shuffle can be broken into other shuffles and the legalizer can
7356 // Normalize the node to match x86 shuffle ops if needed
7433 // Since no target specific shuffle was selected for this generic one,
7458 // several different shuffle types.
8493 SDValue Shuffle = getTargetShuffleNode(X86ISD::PSHUFD, dl, MVT::v4i32,
8496 DAG.getNode(ISD::BITCAST, dl, MVT::v2f64, Shuffle),
8943 // Prepare truncation shuffle mask
10171 // for v4i32 the shuffle mask will be { 0, 1, -1, -1} {2, 3, -1, -1}
11892 // Lower v4i32 mul as 2x shuffle, 2x pmuludq, 2x shuffle.
11910 // Merge the two vectors back together with a shuffle. This expands into 2
15489 /// isShuffleHigh128VectorInsertLow - Checks whether the shuffle node is the
15505 /// isShuffleLow128VectorInsertHigh - Checks whether the shuffle node is the
15521 /// PerformShuffleCombine256 - Performs shuffle combines for 256-bit vectors.
15552 // To match the shuffle mask, the first half of the mask should
15617 /// PerformShuffleCombine - Performs several different shuffle combines.
15649 /// a sequence of vector shuffle operations.
15658 /// specific shuffle of a load can be folded into a single element load.
15706 // If inputs to shuffle are the same for both ops, then allow 2 uses
15727 // If there's a bitcast before the shuffle, check if the load type and
15742 // Create shuffle node taking into account the case that its a unary shuffle
15743 SDValue Shuffle = (UnaryShuffle) ? DAG.getUNDEF(VT) : InVec.getOperand(1);
15744 Shuffle = DAG.getVectorShuffle(InVec.getValueType(), dl,
15745 InVec.getOperand(0), Shuffle,
15747 Shuffle = DAG.getNode(ISD::BITCAST, dl, VT, Shuffle);
15748 return DAG.getNode(ISD::EXTRACT_VECTOR_ELT, dl, N->getValueType(0), Shuffle,
17322 // shuffleshuffle but the
17325 // emit a shuffle and a arithmetic shift.
17327 // during the shuffle phase or after the shuffle.
17384 // We can't shuffle using an illegal type.
17427 // Otherwise we'll shuffle the small elements in the high bits of the
17511 // Optimize trunc store (of multiple scalars) to shuffle and store.
17531 // Create a type on which we perform the shuffle
17542 // Can't shuffle using an illegal type.
17730 // At least one of the operands should be a vector shuffle.
17751 // If LHS is not a shuffle then pretend it is the shuffle
